expirationWarningThreshold

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

expirationWarningThreshold

Tom Scavo
I see that a new expirationWarningThreshold XML attribute will be
introduced in IdP v3.4:
https://wiki.shibboleth.net/confluence/x/8YAOAQ

From the wiki page:

<quote>
expirationWarningThreshold
Duration [default: PT12H]
When a metadata refresh completes, if the live effective metadata root
element's validUntil indicates an expiration time within the specified
duration from the current time, log a warning about the impending
expiration.
</quote>

What exactly is a "completed metadata refresh?" I assume that does not
include an attempted metadata refresh that was short-circuited via
HTTP Conditional GET, correct?

Presumably, an appropriate setting for expirationWarningThreshold will
depend on the values of the other configuration parameters. Suppose a
reloading metadata provider is configured to refresh metadata at least
daily. If validUntil is 14 days in the future, then the
expirationWarningThreshold should be set to PT13D, right?

I'm probably not understanding something since the default value is
just PT12H. That's why I asked about "completed metadata refresh"
initially.

Thanks,

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Brent Putman



On 5/13/17 8:26 PM, Tom Scavo wrote:
What exactly is a "completed metadata refresh?" I assume that does not
include an attempted metadata refresh that was short-circuited via
HTTP Conditional GET, correct?

No, it includes literally every refresh cycle.  The eval is done at the end of the refresh() call, regardless of what did or did not happen during the refresh.

The issue is: https://issues.shibboleth.net/jira/browse/OSJ-200

The main use case that prompted Scott to open the issue is precisely what you mentioned above:  No new metadata bytes were fetched b/c they haven't changed - BUT the metadata is either expired or in danger of expiring "soon".

So now at the end of the refresh we check the live batch root validUntil, and log a WARN on three cases:

1) metadata is already expired or otherwise invalid
2) it's going to expire within the configured expirationWarningThreshold
3) it's going to expire before the next scheduled refresh


Presumably, an appropriate setting for expirationWarningThreshold will
depend on the values of the other configuration parameters. 

I suppose that's true, although I think you also need to factor in knowledge about the "expected" validity duration of the metadata source.


Suppose a
reloading metadata provider is configured to refresh metadata at least
daily. If validUntil is 14 days in the future, then the
expirationWarningThreshold should be set to PT13D, right?

I'm not sure about that.  The point of the WARN log is to have proactive notification of when something goes wrong.  For example, the metadata publisher's process stops working, or the publishing endpoint stops responding, etc.  So the "typical" validUntil isn't really the main concern here.


I'm probably not understanding something since the default value is
just PT12H.

As we discussed there at the end of the Jira issue, there's really probably no good default, since it really depends on how often the resolver refresh runs as well as what the typical validity duration is.

The value really is probably "How much time in advance of a potential expired metadata situation do you want to see a WARN log, so you can intervene and do something about it?"



--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Tom Scavo
On Sat, May 13, 2017 at 11:23 PM, Brent Putman <[hidden email]> wrote:

>
> On 5/13/17 8:26 PM, Tom Scavo wrote:
>>
>> What exactly is a "completed metadata refresh?" I assume that does not
>> include an attempted metadata refresh that was short-circuited via
>> HTTP Conditional GET, correct?
>
> No, it includes literally every refresh cycle.  The eval is done at the end
> of the refresh() call, regardless of what did or did not happen during the
> refresh.

Ah, thanks for clarifying that. That makes much more sense.

> The issue is: https://issues.shibboleth.net/jira/browse/OSJ-200

I see I'm a little late to the party :-)

> The main use case that prompted Scott to open the issue is precisely what
> you mentioned above:  No new metadata bytes were fetched b/c they haven't
> changed - BUT the metadata is either expired or in danger of expiring
> "soon".

Understood.

> So now at the end of the refresh we check the live batch root validUntil,
> and log a WARN on three cases:
>
> 1) metadata is already expired or otherwise invalid
> 2) it's going to expire within the configured expirationWarningThreshold

Those two cases make sense...

> 3) it's going to expire before the next scheduled refresh

but how do you know that?

>> Presumably, an appropriate setting for expirationWarningThreshold will
>> depend on the values of the other configuration parameters.
>
> I suppose that's true

Well, you just said "next scheduled refresh," but that is a configured
event, right?

> although I think you also need to factor in knowledge
> about the "expected" validity duration of the metadata source.

Yeah, I guess so, but that's a bit of a concern. It means that
changing the validity interval on the file may breaking existing
configurations. For example, suppose a federation has the following
characteristics:

Current operation:
Metadata is published on business days (excluding weekends and holidays)
Maximum 5 days between signings
validUntil is 14 days

Future operation:
Metadata is published daily (including weekends and holidays)
Maximum 1 day between signings
validUntil is 7 days

For the current operation, expirationWarningThreshold might be set to
PT7D (or more). For the future operation, PT3D makes sense.

Clearly PT7D won't work for the future operation. Should deployers be
advised to configure PT3D for the current operation so that they don't
have to reconfigure in the future?

>> Suppose a
>> reloading metadata provider is configured to refresh metadata at least
>> daily. If validUntil is 14 days in the future, then the
>> expirationWarningThreshold should be set to PT13D, right?
>
> I'm not sure about that.

Right, I incorrectly assumed that "completed metadata refresh" implied
updated metadata.

> The point of the WARN log is to have proactive
> notification of when something goes wrong.  For example, the metadata
> publisher's process stops working, or the publishing endpoint stops
> responding, etc.  So the "typical" validUntil isn't really the main concern
> here.

Yes, I get that now.

> The value really is probably "How much time in advance of a potential
> expired metadata situation do you want to see a WARN log, so you can
> intervene and do something about it?"

For my particular situation, I don't know the answer to that question.
I need to think about it some more.

Thanks,

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Brent Putman



On 5/14/17 7:15 PM, Tom Scavo wrote:

3) it's going to expire before the next scheduled refresh
but how do you know that?

Each refresh operation computes the date/time for the next refresh and schedules it.  So the resolver knows it because it decided it.


Well, you just said "next scheduled refresh," but that is a configured
event, right?

Well, it's computed, based on both resolver configuration and the metadata's validUntil and cacheDuration.  (None of this is new, nor has it changed in a long time).


Yeah, I guess so, but that's a bit of a concern. It means that
changing the validity interval on the file may breaking existing
configurations. For example, suppose a federation has the following
characteristics:

Well, nothing is going to "break".  This is just about logging a WARN.  And remember that it also logs if the metadata will expire before the next refresh, so that takes care of the case of some assumptions changing.


For the current operation, expirationWarningThreshold might be set to
PT7D (or more). For the future operation, PT3D makes sense.


Well, I personally think both of those are way too long.  I think probably 24 hours is the most that makes sense - unless you really don't want to get notified and have to do something on a weekend, in which case maybe it's longer. (But I don't see how IT devops people can really get away with that these days...)   But each person will have their own requirements.  It's really more of an operational question of how paranoid you want to be.


Clearly PT7D won't work for the future operation. Should deployers be
advised to configure PT3D for the current operation so that they don't
have to reconfigure in the future?


Maybe. But if fundamental assumptions about the metadata publishing change, then I think many of the config params (e.g. min- and maxCacheDuration) might have to be revisited by the deployer.  We can't deal with everything.



--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Tom Scavo
I now understand how the expirationWarningThreshold works (thanks
Brent). I'm still not clear on the intended use case, however, so let
me iterate one more time.

On Sun, May 14, 2017 at 7:30 PM, Brent Putman <[hidden email]> wrote:

>
> On 5/14/17 7:15 PM, Tom Scavo wrote:
>>
>> For the current operation, expirationWarningThreshold might be set to
>> PT7D (or more). For the future operation, PT3D makes sense.
>
> Well, I personally think both of those are way too long.  I think probably
> 24 hours is the most that makes sense - unless you really don't want to get
> notified and have to do something on a weekend, in which case maybe it's
> longer. (But I don't see how IT devops people can really get away with that
> these days...)   But each person will have their own requirements.  It's
> really more of an operational question of how paranoid you want to be.

At this point, I'm not entirely sure what advice to give if this comes
up on the users list, but there's one thing I'm sure about: a metadata
provider configured to refresh federation metadata with
expirationWarningThreshold="PT24H" won't do anybody much good since a
deployer doesn't directly control the validUntil attribute on
federation metadata. Once a warning is issued, the metadata will
certainly be rejected by the software in 24 hrs unless the federation
operator intervenes. So it seems to me a deployer will want to know
well enough ahead of time (or not at all) so that the federation
operator can be alerted.

Am I missing something?

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: expirationWarningThreshold

Cantor, Scott E.
> I now understand how the expirationWarningThreshold works (thanks
> Brent). I'm still not clear on the intended use case, however, so let
> me iterate one more time.

The use case for me is my local metadata feeds that my customer(s) keep breaking on me and then complain when my system just consumes their expired metadata. The warning gives me a clearer signal they broke it and some advance notice if they do.

In the current code, you get *no* indication anything is up until it expires and then you just get errors of the "no metadata found" sort, which make you think it's just people using the wrong entityID or something.

> At this point, I'm not entirely sure what advice to give if this comes
> up on the users list, but there's one thing I'm sure about: a metadata
> provider configured to refresh federation metadata with
> expirationWarningThreshold="PT24H" won't do anybody much good since a
> deployer doesn't directly control the validUntil attribute on
> federation metadata. Once a warning is issued, the metadata will
> certainly be rejected by the software in 24 hrs unless the federation
> operator intervenes. So it seems to me a deployer will want to know
> well enough ahead of time (or not at all) so that the federation
> operator can be alerted.

Yes, s/federation operator/metadata source

There's no default value with any meaning, the value is dependent on the metadata source and what you're going to do if it barks. It isn't deriveable from any other setting directly, though it's obviously related to the validity period.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Brent Putman



On 5/16/17 3:07 PM, Cantor, Scott wrote:

At this point, I'm not entirely sure what advice to give if this comes
up on the users list, but there's one thing I'm sure about: a metadata
provider configured to refresh federation metadata with
expirationWarningThreshold="PT24H" won't do anybody much good since a
deployer doesn't directly control the validUntil attribute on
federation metadata. 

No, they don't control. But the point I was trying to make is that this setting is not about what happens under normal conditions, but rather under abnormal conditions.  It's not something you can compute *directly* from knowledge of how the metadata source populates the validUntil (although that knowledge is obviously relevant).  Because the very use case is "what happens when all of that breaks".  So the most I can say is that it's how far in advance of a potential problem do you want to be notified so you can do something about it.  If you think it would take greater than 24hrs to get your metadata source or federation operator to fix things, then maybe it is indeed greater than 24hrs.


Once a warning is issued, the metadata will
certainly be rejected by the software in 24 hrs unless the federation
operator intervenes. 

Well to be clear, it's worse than that.  It might expire in 1 minute.  Or it might expire in 23 hrs 59 mins.  All you are doing is setting the threshold (window) within which you would want to be notified.  So it also sort of matters how frequently your refresh cycle runs.


So it seems to me a deployer will want to know
well enough ahead of time (or not at all) so that the federation
operator can be alerted.

Yes.  Which is why it's difficult to give precise guidance on what the value should be.  It's very dependent on a bunch of interrelated factors.

Yes, s/federation operator/metadata source

There's no default value with any meaning, the value is dependent on the metadata source and what you're going to do if it barks. It isn't deriveable from any other setting directly, though it's obviously related to the validity period.

Yes, exactly.

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Michael A Grady

up on the users list, but there's one thing I'm sure about: a metadata
provider configured to refresh federation metadata with
expirationWarningThreshold="PT24H" won't do anybody much good since a
deployer doesn't directly control the validUntil attribute on
federation metadata. 

No, they don't control. But the point I was trying to make is that this setting is not about what happens under normal conditions, but rather under abnormal conditions.  It's not something you can compute *directly* from knowledge of how the metadata source populates the validUntil (although that knowledge is obviously relevant).  Because the very use case is "what happens when all of that breaks".  So the most I can say is that it's how far in advance of a potential problem do you want to be notified so you can do something about it.  If you think it would take greater than 24hrs to get your metadata source or federation operator to fix things, then maybe it is indeed greater than 24hrs.


I'm not sure why you think setting this value to more than 24 hours wouldn't make sense. Take the InCommon metadata, for example. I know it has a valid period of 2 weeks, and I know that just about every business day that metadata changes, and should be (if I have recommended refresh settings) therefore refreshed about once a business day. So if see that the validUntil of my copy is less than a week, then something is going wrong. it doesn't take getting to within 24 hours or less of it expiring to know I have a problem and things aren't working as I'd expect.

--
Michael A. Grady
IAM Architect, Unicon, Inc.




--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Brent Putman



On 5/16/17 6:31 PM, Michael A Grady wrote:


I'm not sure why you think setting this value to more than 24 hours wouldn't make sense.

That number was mostly driven by  1) the notion that trying to reduce false alarms is good 2) an assumption that if you notice a problem, the metadata source can fix quickly after being notified (less than a day). If either of those isn't true, then maybe > 24 hours does make sense. 


Take the InCommon metadata, for example. I know it has a valid period of 2 weeks, and I know that just about every business day that metadata changes, and should be (if I have recommended refresh settings) therefore refreshed about once a business day. So if see that the validUntil of my copy is less than a week, then something is going wrong. it doesn't take getting to within 24 hours or less of it expiring to know I have a problem and things aren't working as I'd expect.

Sure.  If you want that much notice, then you can set to 7 days or whatever.  But there could be legitimate reasons why the "expected" update schedule changes for some period of time (i.e. no updates to publish), and then corrects itself well within days of expiration.  You would get a false alarm in that case with a lengthy threshold.  If you're ok with that, then that's fine.

As I said somewhere in this thread, it really comes down to how paranoid you want to be.  More paranoid = greater chance of false alarm.  And if you have a 7 day threshold and the warning fires, you have to decide whether you go and pester the metadata source immediately, or wait a few days to see if it resolves on its own.

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Tom Scavo
A couple of additional questions about this (probably for Brent):

What happens if there is no validUntil XML attribute on the root
element? I would assume the expirationWarningThreshold setting is
ignored in this case. Correct?

Would it make sense to treat expired metadata differently than
metadata that exceeds the threshold (but is not expired)? If expired
metadata is expunged from the system (which of course is the right
thing to do), I would think this deserves serious consideration,
perhaps an error message instead of a warning?

Tom


On Tue, May 16, 2017 at 7:10 PM, Brent Putman <[hidden email]> wrote:

>
>
> On 5/16/17 6:31 PM, Michael A Grady wrote:
>
>
> I'm not sure why you think setting this value to more than 24 hours wouldn't
> make sense.
>
>
> That number was mostly driven by  1) the notion that trying to reduce false
> alarms is good 2) an assumption that if you notice a problem, the metadata
> source can fix quickly after being notified (less than a day). If either of
> those isn't true, then maybe > 24 hours does make sense.
>
>
> Take the InCommon metadata, for example. I know it has a valid period of 2
> weeks, and I know that just about every business day that metadata changes,
> and should be (if I have recommended refresh settings) therefore refreshed
> about once a business day. So if see that the validUntil of my copy is less
> than a week, then something is going wrong. it doesn't take getting to
> within 24 hours or less of it expiring to know I have a problem and things
> aren't working as I'd expect.
>
>
> Sure.  If you want that much notice, then you can set to 7 days or whatever.
> But there could be legitimate reasons why the "expected" update schedule
> changes for some period of time (i.e. no updates to publish), and then
> corrects itself well within days of expiration.  You would get a false alarm
> in that case with a lengthy threshold.  If you're ok with that, then that's
> fine.
>
> As I said somewhere in this thread, it really comes down to how paranoid you
> want to be.  More paranoid = greater chance of false alarm.  And if you have
> a 7 day threshold and the warning fires, you have to decide whether you go
> and pester the metadata source immediately, or wait a few days to see if it
> resolves on its own.
>
> --
> To unsubscribe from this list send an email to
> [hidden email]
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Brent Putman



On 5/19/17 10:14 AM, Tom Scavo wrote:

What happens if there is no validUntil XML attribute on the root
element? I would assume the expirationWarningThreshold setting is
ignored in this case. Correct?

Yes, it is ignored.

Related: There are obvious caveats about this sort of thing wrt root element validUntil, since of course one could have an aggregate EntitiesDescriptor which itself doesn't contain a validUntil, but the child Entity- and EntitiesDescriptors do.

So this new config param doesn't attempt to address those cases, only the vast majority case of a root element validUntil.


Would it make sense to treat expired metadata differently than
metadata that exceeds the threshold (but is not expired)? If expired
metadata is expunged from the system (which of course is the right
thing to do), I would think this deserves serious consideration,
perhaps an error message instead of a warning?

Well as far as logging an ERROR vs a WARN, our convention is to usually only log an ERROR when there's something seriously wrong with the system at a fundamental level.  Since this is really about "data", I'm not sure that this qualifies.  But if the consensus of the other devs is otherwise, it's certainly easy to change. The logging call for the already-expired case is separate from the pending expiration cases.

We in fact do not actually literally expunge expired metadata, but we don't return it from a metadata resolve call.  Each EntityDescriptor candidate which is resolved is checked individually at runtime for validity, and if invalid and isRequireValidMetadata==true, then it's thrown out from the result set.

   


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Tom Scavo
On Fri, May 19, 2017 at 4:11 PM, Brent Putman <[hidden email]> wrote:

>
> On 5/19/17 10:14 AM, Tom Scavo wrote:
>>
>> What happens if there is no validUntil XML attribute on the root
>> element? I would assume the expirationWarningThreshold setting is
>> ignored in this case. Correct?
>
> Yes, it is ignored.
>
> Related: There are obvious caveats about this sort of thing wrt root element
> validUntil, since of course one could have an aggregate EntitiesDescriptor
> which itself doesn't contain a validUntil, but the child Entity- and
> EntitiesDescriptors do.
>
> So this new config param doesn't attempt to address those cases, only the
> vast majority case of a root element validUntil.

Thanks for clarifying. All of that makes sense. It needs to be
documented, however.

>> Would it make sense to treat expired metadata differently than
>> metadata that exceeds the threshold (but is not expired)? If expired
>> metadata is expunged from the system (which of course is the right
>> thing to do), I would think this deserves serious consideration,
>> perhaps an error message instead of a warning?
>
> Well as far as logging an ERROR vs a WARN, our convention is to usually only
> log an ERROR when there's something seriously wrong with the system at a
> fundamental level.  Since this is really about "data", I'm not sure that
> this qualifies.  But if the consensus of the other devs is otherwise, it's
> certainly easy to change. The logging call for the already-expired case is
> separate from the pending expiration cases.

Okay, let's see what others think about this.

> We in fact do not actually literally expunge expired metadata, but we don't
> return it from a metadata resolve call.

This is not made clear in all the documentation I've read. Of course I
could have missed something.

> Each EntityDescriptor candidate
> which is resolved is checked individually at runtime for validity, and if
> invalid and isRequireValidMetadata==true, then it's thrown out from the
> result set.

That, too, is not clear by reading the documentation. I was under the
impression that invalid metadata was ignored, full stop. Also, what is
isRequireValidMetadata? I can't find that in the docs.

Thanks,

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Brent Putman



On 5/19/17 4:20 PM, Tom Scavo wrote:

      
Thanks for clarifying. All of that makes sense. It needs to be
documented, however.

I added a parenthetical qualifier to the expirationWarningThreshold docs on the wiki. 

I don't know how one would document any of the other.  It's not about Shibboleth software, it's about general metadata concepts.



We in fact do not actually literally expunge expired metadata, but we don't
return it from a metadata resolve call.
This is not made clear in all the documentation I've read. Of course I
could have missed something.

The not expunging part isn't.  But it's an internal implementation detail, so I don't know why a deployer would care.  The necessity of the entity to be valid to be used is documented in the requireValidMetadata attribute. 



That, too, is not clear by reading the documentation.

It's in the requireValidMetadata attribute.

 I was under the
impression that invalid metadata was ignored, full stop.

As far as the calling code is concerned, it is ignored, full stop.   What I mentioned is the internal impl detail of the resolvers.  This is the dev list, so I'm mentioning details of the software that would be irrelevant to an end-user deployer.


 Also, what is
isRequireValidMetadata? I can't find that in the docs.

That's the Java method name, corresponds to the resolver config attribute requireValidMetadata:

https://wiki.shibboleth.net/confluence/display/IDP30/MetadataConfiguration#MetadataConfiguration-Attributes

It's been there since v2.2.



--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Tom Scavo
On Fri, May 19, 2017 at 4:37 PM, Brent Putman <[hidden email]> wrote:
>
> On 5/19/17 4:20 PM, Tom Scavo wrote:
>>
>> Thanks for clarifying. All of that makes sense. It needs to be
>> documented, however.
>
> I added a parenthetical qualifier to the expirationWarningThreshold docs on
> the wiki.

Thanks Brent. I modified the text yet again. Please review and edit accordingly.

> I don't know how one would document any of the other.  It's not about
> Shibboleth software, it's about general metadata concepts.

Not all SAML metadata concepts are well documented. The SAML metadata
spec says very little about validUntil. I don't recall if the
implementation profile we wrote says anything further about it.

> The necessity of the entity to
> be valid to be used is documented in the requireValidMetadata attribute.
>
>> That, too, is not clear by reading the documentation.
>
> It's in the requireValidMetadata attribute.

Okay, now I'm more confused. If the requireValidMetadata attribute is
about *usage* of cached metadata, that's news to me.

In any case, I edited the text for requireValidMetadata so that it
corresponds to my understanding (which is probably wrong). Please
review and edit as necessary.

>> Also, what is
>> isRequireValidMetadata? I can't find that in the docs.
>
> That's the Java method name, corresponds to the resolver config attribute
> requireValidMetadata:
>
> https://wiki.shibboleth.net/confluence/display/IDP30/MetadataConfiguration#MetadataConfiguration-Attributes
>
> It's been there since v2.2.

Yes, I see that now, thanks for reminding me, but I'm afraid this is
getting overly complicated. We now have:

1. MetadataProvider/@requireValidMetadata
2. MetadataProvider/@expirationWarningThreshold
3. MetadataProvider/MetadataFilter[@xsi:type="RequiredValidUntil"]

If we were starting from scratch, would it make sense to expose the
latter two configurations only? AFAICT, the first one isn't really an
option. I claim an implementation doesn't have the option of ignoring
the validUntil attribute. Do you agree?

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: expirationWarningThreshold

Cantor, Scott E.
> If we were starting from scratch, would it make sense to expose the
> latter two configurations only? AFAICT, the first one isn't really an
> option. I claim an implementation doesn't have the option of ignoring
> the validUntil attribute. Do you agree?

No, the low level resolver code is available for general metadata processing that may have a reason to be seeing all metadata, expired or not.

Whether we document or mention the setting in the Shibboleth context is a different question, but as an option it's necessary.

As I keep saying, nobody who doesn't understand expirationWarningThreshold has any reason to use it. I'm inclined to say we should reverse the default change and put it back at 0 or something really small so that it doesn't impact anybody who doesn't choose to set it.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Brent Putman
In reply to this post by Tom Scavo



On 5/21/17 11:20 AM, Tom Scavo wrote:
Okay, now I'm more confused. If the requireValidMetadata attribute is
about *usage* of cached metadata, that's news to me.

Well, it's been there for over a decade, and this code hasn't fundamentally changed in all that time.
 


In any case, I edited the text for requireValidMetadata so that it
corresponds to my understanding (which is probably wrong). Please
review and edit as necessary.

The edits aren't correct.  The requireValidMetadata is not just about the validUntil on the root element.  1) You're forgetting that more than the root element can carry validUntil.  When processing a candidate EntityDescriptor, that element's validUntil as well as all ancestor element validUntils are checked.  2) Strictly speaking, in a software sense, this is about descriptor element validity more generally, as the original docs noted.  That could be about more than validUntil checks (although currently that is all that is checked, as was originally noted in the docs).  I'll revert the docs back, with a little bit of rewording to make more clear.

FYI, in terms of symmetry, there is a similar config flag and logic on the RoleDescriptorResolver impl.  Remember that RoleDescriptor can carry validUntil as well.


If we were starting from scratch, would it make sense to expose the
latter two configurations only? AFAICT, the first one isn't really an
option. I claim an implementation doesn't have the option of ignoring
the validUntil attribute. Do you agree?

Like Scott, I disagree.  Aside from other low-level software use cases where you might really want to process "invalid" metadata, there's also the case of a broken metadata source that's taking awhile to get fixed.  If you're an IdP and you have to tell your users that they absolutely can't use some service because of some stale/expired metadata, due to hardcoded and un-changeable software behaviour, then that might be bad.  This flag puts that decision into the hands of the IdP deployer.  If they are OK with temporarily disabling that check while the situation with the metadata source gets sorted out, then that ought to be their decision.



--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Brent Putman
In reply to this post by Cantor, Scott E.



On 5/22/17 10:04 AM, Cantor, Scott wrote:
As I keep saying, nobody who doesn't understand expirationWarningThreshold has any reason to use it. 

Agree.

I'm inclined to say we should reverse the default change and put it back at 0 or something really small so that it doesn't impact anybody who doesn't choose to set it.

As we have said in the thread and issue, there really isn't a universally good default, it's highly individual.  So if the consensus is that we default it down to some very low number (1 ms, etc) so it never really comes into play, and therefore requires explicit configuration, then that's OK by me.

Remember that even in that case, if the metadata will expire before the next refresh, then you still get a log warning.

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: expirationWarningThreshold

Cantor, Scott E.
> As we have said in the thread and issue, there really isn't a universally good
> default, it's highly individual.  So if the consensus is that we default it down to
> some very low number (1 ms, etc) so it never really comes into play, and
> therefore requires explicit configuration, then that's OK by me.

Is there a need for it to be on by default? Could it just default to 0 and do nothing until the actual expiration? I haven't looked at the new code so I'm not really up on how it works.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Brent Putman



On 5/22/17 1:47 PM, Cantor, Scott wrote:
As we have said in the thread and issue, there really isn't a universally good
default, it's highly individual.  So if the consensus is that we default it down to
some very low number (1 ms, etc) so it never really comes into play, and
therefore requires explicit configuration, then that's OK by me.
Is there a need for it to be on by default? Could it just default to 0 and do nothing until the actual expiration? I haven't looked at the new code so I'm not really up on how it works.

You're right, that's probably the best thing to do, and easy enough to do.  I will make that change.

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: expirationWarningThreshold

Tom Scavo
In reply to this post by Brent Putman
On Mon, May 22, 2017 at 1:40 PM, Brent Putman <[hidden email]> wrote:

>
> Aside from other low-level software use cases where
> you might really want to process "invalid" metadata, there's also the case
> of a broken metadata source that's taking awhile to get fixed.  If you're an
> IdP and you have to tell your users that they absolutely can't use some
> service because of some stale/expired metadata, due to hardcoded and
> un-changeable software behaviour, then that might be bad.  This flag puts
> that decision into the hands of the IdP deployer.  If they are OK with
> temporarily disabling that check while the situation with the metadata
> source gets sorted out, then that ought to be their decision.

I didn't know that the requireValidMetadata flag could be used in this
way. This should probably be documented.

Tom
--
To unsubscribe from this list send an email to [hidden email]
12