reconciling the Reloading Attributes with the Dynamic Attributes

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

reconciling the Reloading Attributes with the Dynamic Attributes

Tom Scavo
I'm trying to reconcile the Reloading Attributes with the Dynamic
Attributes on a metadata provider. [1] Basically I have two questions:

1) What is the effect of the metadata's @cacheDuration attribute on
the operation of a reloading provider (such as the
FileBackedHTTPMetadataProvider)?

2) What is the effect of the metadata's @validUntil attribute on the
operation of a dynamic provider (such as the HTTPMetadataProvider)?

The answer to the first question would appear to be "none" since a
FileBackedHTTPMetadataProvider is a non-caching metadata provider. Is
this correct?.

The answer to the second question depends on the value of the
requireValidMetadata attribute but why then is the
expirationWarningThreshold attribute confined to the reloading
attributes?

Thanks,

Tom

[1] https://wiki.shibboleth.net/confluence/x/8YAOAQ
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: reconciling the Reloading Attributes with the Dynamic Attributes

Cantor, Scott E.
> 1) What is the effect of the metadata's @cacheDuration attribute on the
> operation of a reloading provider (such as the
> FileBackedHTTPMetadataProvider)?

It sets bounds on the refresh timer's behavior, I believe, so it can trigger earlier refresh attempts if it's small, within the bounds of the min/max settings. It's a signal to try for new data more often, which seems logical to me.

> 2) What is the effect of the metadata's @validUntil attribute on the
> operation of a dynamic provider (such as the HTTPMetadataProvider)?

The dynamic provider has a bit of "duality" with how it treats validity. In the dynamic layer, it actually applies both cacheDuration and validUntil at the same time to compute upper bounds on what it calls "validity", which isn't really what you think of as metadata being valid or not, it's just refresh behavior since the whole idea is that the original source should be able to supply the latest metadata anyway. So either attribute can reduce the refresh delay and cause it to query more often.

But at the top layer, it's still looking only at validUntil to decide on validity, as it should.

> The answer to the first question would appear to be "none" since a
> FileBackedHTTPMetadataProvider is a non-caching metadata provider. Is this
> correct?.

I don't think so.

> The answer to the second question depends on the value of the
> requireValidMetadata attribute but why then is the
> expirationWarningThreshold attribute confined to the reloading attributes?

It does not depend on that and I don't know anything about the other attribute, so I can't say.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: reconciling the Reloading Attributes with the Dynamic Attributes

Tom Scavo
On Tue, May 8, 2018 at 12:54 PM, Cantor, Scott <[hidden email]> wrote:
>> 1) What is the effect of the metadata's @cacheDuration attribute on the
>> operation of a reloading provider (such as the
>> FileBackedHTTPMetadataProvider)?
>
> It sets bounds on the refresh timer's behavior, I believe, so it can trigger earlier refresh attempts if it's small, within the bounds of the min/max settings. It's a signal to try for new data more often, which seems logical to me.

Thanks, that makes sense. Is the same true in the case of a dynamic
provider? If so, why are the attributes called
minRefreshDelay/maxRefreshDelay in the one case and
minCacheDuration/maxCacheDuration in the other?

Btw, note that the refreshDelayFactor attribute has exactly the same
description in both cases. That description really only makes sense
for a reloading provider. In the case of a dynamic provider, the
description is very confusing.

I'm guessing the reason for these inconsistencies is purely
historical. If so, now is a good time to rationalize the attribute
names (since deprecated features are being rooted out as we speak).

>> The answer to the first question would appear to be "none" since a
>> FileBackedHTTPMetadataProvider is a non-caching metadata provider. Is this
>> correct?.
>
> I don't think so.

I believe you. The point I'm trying to make is that httpCaching="none"
in the case of FileBackedHTTPMetadataProvider is an apparent
contradiction.

>> The answer to the second question depends on the value of the
>> requireValidMetadata attribute but why then is the
>> expirationWarningThreshold attribute confined to the reloading attributes?
>
> It does not depend on that and I don't know anything about the other attribute, so I can't say.

The requireValidMetadata attribute is a Common Attribute so it should
apply to a dynamic provider, in which case the
expirationWarningThreshold attribute (which itself is misnamed [1])
should also apply. That seems like a bug.

Tom

[1] https://issues.shibboleth.net/jira/browse/OSJ-236
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: reconciling the Reloading Attributes with the Dynamic Attributes

Cantor, Scott E.
> Thanks, that makes sense. Is the same true in the case of a dynamic
> provider? If so, why are the attributes called
> minRefreshDelay/maxRefreshDelay in the one case and
> minCacheDuration/maxCacheDuration in the other?

I believe the dynamic ones don't refresh per se, they just dump their cached data or mark it to indicate a new copy should be fetched. The names seem to match that to me, so I think they're fine.

> Btw, note that the refreshDelayFactor attribute has exactly the same
> description in both cases. That description really only makes sense for a
> reloading provider. In the case of a dynamic provider, the description is very
> confusing.

It's doing the same thing, modifying the caclulated value that governs how often it refreshes, it's just the underlying mechanics that are a bit different. I'm sure the text is just a cut and paste.

> I believe you. The point I'm trying to make is that httpCaching="none"
> in the case of FileBackedHTTPMetadataProvider is an apparent contradiction.

It isn't, but then I know what the code is doing.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: reconciling the Reloading Attributes with the Dynamic Attributes

Tom Scavo
On Tue, May 8, 2018 at 3:11 PM, Cantor, Scott <[hidden email]> wrote:
>>
>> Btw, note that the refreshDelayFactor attribute has exactly the same
>> description in both cases. That description really only makes sense for a
>> reloading provider. In the case of a dynamic provider, the description is very
>> confusing.
>
> It's doing the same thing, modifying the caclulated value that governs how often it refreshes

But you said "the dynamic ones don't refresh per se" so I'm confused.
Either the refreshDelayFactor attribute on the dynamic provider is
misnamed or the minCacheDuration/maxCacheDuration attributes are
misnamed. There's a glaring mismatch here.

There's a significant optimization lurking just under the surface
(which is why I'm probing the details). If you deprecate
InlineMetadataProvider (no big loss), you're left with just two groups
of metadata providers: the reloading providers and the dynamic
providers. In that case, a handful of attributes common to both may be
elevated to the Common Attributes.

>> I believe you. The point I'm trying to make is that httpCaching="none"
>> in the case of FileBackedHTTPMetadataProvider is an apparent contradiction.
>
> It isn't, but then I know what the code is doing.

What if all four of the HTTP Caching Attributes were deprecated? Since
caching is covered quite nicely in the HttpClientConfiguration topic,
[1] the HTTP Caching Attributes are merely convenience attributes. In
the vast majority of cases, they probably shouldn't be changed anyway,
so they turn out to be less than convenient, I think.

Tom

[1] https://wiki.shibboleth.net/confluence/x/PgEOAg
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: reconciling the Reloading Attributes with the Dynamic Attributes

Cantor, Scott E.
> But you said "the dynamic ones don't refresh per se" so I'm confused.

I was speaking loosely, refresh in the sense of when it decides to go ask again.

> Either the refreshDelayFactor attribute on the dynamic provider is misnamed
> or the minCacheDuration/maxCacheDuration attributes are misnamed.
> There's a glaring mismatch here.

You'll have to take that up with Brent, I don't get much into arguments over names, it's all the same to me as long as it's reasonable. What it's doing in the end is still refreshing, it's just the mechanics that are different. I didn't find it that confusing.

> What if all four of the HTTP Caching Attributes were deprecated? Since
> caching is covered quite nicely in the HttpClientConfiguration topic, [1] the
> HTTP Caching Attributes are merely convenience attributes. In the vast
> majority of cases, they probably shouldn't be changed anyway, so they turn
> out to be less than convenient, I think.

I really am not all that familiar with them but auto-wiring up HttpClient objects with default settings is indeed not a terribly useful thing in practice and likely causes more problems than it solves as soon as you have to override a setting they don't cover, so that's probably a reasonable idea, given that there's a bit of code behind them that could be thrown out at some point.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: reconciling the Reloading Attributes with the Dynamic Attributes

Rod Widdowson
In reply to this post by Tom Scavo
> now is a good time to rationalize the attribute
> names (since deprecated features are being rooted out as we speak).

Yes, but there is a baby/bathwater risk here too.

R

--
To unsubscribe from this list send an email to [hidden email]