include SignatureValidation filter with FileBackedHTTPMetadataProvider

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

include SignatureValidation filter with FileBackedHTTPMetadataProvider

Tom Scavo
Please review this added security consideration:

HTTPMetadataProviders
https://wiki.shibboleth.net/confluence/x/kQInAQ

Thanks,

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Cantor, Scott E.
> Please review this added security consideration:

I don't really understand what you're trying to get at with it.

If the file is signed, then you should use the filter, if not, not. The backup file is not something that should be factored into the determination, and the SP now doesn't generally even get told to do the check on the backup copy anymore (the IdP doesn't have that feature yet but probably will in the future).

And, really, nobody uses the HTTPMetadataProvider to begin with (should we even be exposing it?), but if you did, you certainly wouldn't omit the signature check if the file was signed.

Just not sure what this is trying to say, but it shouldn't say this.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Tom Scavo
On Tue, Dec 19, 2017 at 11:10 AM, Cantor, Scott <[hidden email]> wrote:
>
> If the file is signed, then you should use the filter, if not, not.

Yes of course. Basically what I'm trying to say is: If the file is NOT
signed, you should not use FileBackedHTTPMetadataProvider.

> And, really, nobody uses the HTTPMetadataProvider to begin with (should we even be exposing it?), but if you did, you certainly wouldn't omit the signature check if the file was signed.

Sure, I didn't mean to imply otherwise. If the file is not signed,
presumably TLS is providing security, in which case use
HTTPMetadataProvider (not FileBackedHTTPMetadataProvider).

Remember when the SP started to have trouble verifying the signature
on large files? Deployers were advised to disable signature
verification upon startup, assuming they could live with the loss of
security.

Same here. If the file is NOT signed, don't use
FileBackedHTTPMetadataProvider since there is no integrity check at
startup.

I could be missing something, I guess. I'm happy to revert the edit if
that turns out to be the case.

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Cantor, Scott E.
> Yes of course. Basically what I'm trying to say is: If the file is NOT
> signed, you should not use FileBackedHTTPMetadataProvider.

I didn't really get that from what you wrote, but I'm not sure as a project we're taking the "TLS sucks" position, whatever my personal feelings, that's for others to decide. We provide extremely (as in the most anywhere) powerful control over TLS just like the SP does, so we intend that both be possible.

> Sure, I didn't mean to imply otherwise. If the file is not signed,
> presumably TLS is providing security, in which case use
> HTTPMetadataProvider (not FileBackedHTTPMetadataProvider).

The difference is one backs up, the other doesn't. There is nothing there that relates to the trust model used, you can use either with either.

> Same here. If the file is NOT signed, don't use
> FileBackedHTTPMetadataProvider since there is no integrity check at
> startup.

There is, though, and the IdP doesn't support skipping it yet.

> I could be missing something, I guess. I'm happy to revert the edit if
> that turns out to be the case.

Please revert it. I still don't know what you mean to say, but at present that text just doesn't make sense.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Tom Scavo
On Tue, Dec 19, 2017 at 12:31 PM, Cantor, Scott <[hidden email]> wrote:
>
> Please revert it.

Done. Sorry for the false start.

> I still don't know what you mean to say...

If the metadata is NOT signed, do not use FileBackedHTTPMetadataProvider.

Why? Because if you do, there's no security upon startup.

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Cantor, Scott E.
> If the metadata is NOT signed, do not use FileBackedHTTPMetadataProvider.
>
> Why? Because if you do, there's no security upon startup.

Except that is *not* true.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Tom Scavo
On Tue, Dec 19, 2017 at 12:57 PM, Cantor, Scott <[hidden email]> wrote:
>> If the metadata is NOT signed, do not use FileBackedHTTPMetadataProvider.
>>
>> Why? Because if you do, there's no security upon startup.
>
> Except that is *not* true.

If the metadata is not signed, there's no way to ensure the integrity
of a backup file upon startup. What am I missing?

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Rod Widdowson

>
> . What am I missing?
>

Well there’s an assumption that you own the security of your filesystem and hence cache.  So if it was safe when it was downloaded it’s still safe (or timed out in which case it won’t load for another reason).

But even to be having this discussion you need to have turned off the signing check on restart which is only possible on the sp and only since very recently (2.6 ?)

/r
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Cantor, Scott E.
In reply to this post by Tom Scavo
> If the metadata is not signed, there's no way to ensure the integrity
> of a backup file upon startup. What am I missing?

Backup files, just like files loaded explicitly by hand in the many ways supported, are presumed to be there as a result of deliberate action or previously secured source. If you don't trust your own file system, you are in deep trouble. It wouldn't have occurred to me to have to point that out but if that's what the confusion is, then that's the missing note.

Simply having the option to skip the verification of the signature, as the SP has, is the reason what you're saying isn't true from the perspective of the software. You can skip it under the presumption that the backup can only get there "safely".

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Tom Scavo
On Tue, Dec 19, 2017 at 1:13 PM, Cantor, Scott <[hidden email]> wrote:
>
> Simply having the option to skip the verification of the signature, as the SP has, is the reason what you're saying isn't true from the perspective of the software. You can skip it under the presumption that the backup can only get there "safely".

There are two separate cases to consider:

If the metadata is signed, and a SignatureValidation filter is
included, the software defaults correctly IMO. If the deployer wishes
to turn off verification of the backup file at startup, s/he can do so
by performing an explicit config action.

OTOH, if the metadata is not signed, what should the default action of
the software be? I believe it should go out to the network by default.
If the deployer makes an explicit decision to grab a locally trusted
file instead, then that's fine. All I'm saying is that shouldn't be
the default behavior of the software (which is it if the deployer
happens to use FileBackedHTTPMetadataProvider).

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Cantor, Scott E.
On 12/19/17, 4:01 PM, "dev on behalf of Tom Scavo" <[hidden email] on behalf of [hidden email]> wrote:

> OTOH, if the metadata is not signed, what should the default action of
> the software be? I believe it should go out to the network by default.
> If the deployer makes an explicit decision to grab a locally trusted
> file instead, then that's fine. All I'm saying is that shouldn't be
> the default behavior of the software (which is it if the deployer
> happens to use FileBackedHTTPMetadataProvider).

We don't agree with that. At least I don't, I guess the rest of the team can speak for themselves, but Brent presumably wouldn't have implemented it that way if he felt it was incorrect to do so.

-- Scott


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Rod Widdowson


>>
>> OTOH, if the metadata is not signed, what should the default action of
>> the software be? I believe it should go out to the network by default.
>>

Presumably because although you were prepared to trust it yesterday, you are not prepared to trust it today? Indeed today you want to trust something else (presumably unsigned)?  I guess that this has something to do with the timing of the transitive nature of TLS trust?

R
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Brent Putman
In reply to this post by Cantor, Scott E.



On 12/19/17 4:31 PM, Cantor, Scott wrote:
On 12/19/17, 4:01 PM, "dev on behalf of Tom Scavo" [hidden email] wrote:

OTOH, if the metadata is not signed, what should the default action of
the software be? I believe it should go out to the network by default.
If the deployer makes an explicit decision to grab a locally trusted
file instead, then that's fine. All I'm saying is that shouldn't be
the default behavior of the software (which is it if the deployer
happens to use FileBackedHTTPMetadataProvider).
We don't agree with that. At least I don't, I guess the rest of the team can speak for themselves, but Brent presumably wouldn't have implemented it that way if he felt it was incorrect to do so.


Although I didn't implement this originally (Chad did), I agree with how it works currently, and with what Scott said earlier in the thread.   The backup file is a local cache of data that you've already fetched and trusted, and is there merely to optimize things when you restart/refresh the provider, as well as provide a backup against the remote source being down when you restart/refresh.  It's implicitly trusted.  It has to be, you have to be able to trust your local system, or it's game over.

I really do not understand the concern about whether it was signed or not, vs the fact that at some particular and arbitrary time, a human (or cron etc) decides to restart/refresh the provider or the IdP.  They just seem completely orthogonal to me.

Consider:  A remote source has a validUntil that is 24 hours.  Consider the case of one file-backed provider that fetches that and isn't restarted, so that you have the same metadata and trust it for the 24 hour duration.  Consider a second file-backed provider that fetches the same, but is restarted/refreshed every hour or two.  Is the latter somehow different vis-a-vis security and trust?  Should it have to re-fetch and re-validate the data merely because a human chose to restart/refresh?  IMHO no.  The restart has nothing to do with how/why you trust that metadata, or the trust in the mechanism by which it is obtained.

If it makes you feel any better, the current behavior of the file-backed provider in the IdP is that it loads the backup file in the foreground thread, and then immediately schedules a background thread refresh attempt for a few seconds in the future (duration is also configurable). So the issue that you are apparently concerned with is only going to have a window of a few seconds.  Assuming the fetch is successful, that is - if it's not, it will keep the data loaded from the backup file, which is the main point of the file-backed provider (in addition to optimizing the startup).


 

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Cantor, Scott E.
On 12/20/17, 2:03 PM, "dev on behalf of Brent Putman" <[hidden email] on behalf of [hidden email]> wrote:

> Although I didn't implement this originally (Chad did),

The part I thought you did was the "load initially from backup and then do a refresh", which I had the impression was a recent change we made in response to the performance problems. That seems to be the behavior under discussion as you alluded to later.

-- Scott
 


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Brent Putman



On 12/20/17 1:07 PM, Cantor, Scott wrote:
On 12/20/17, 2:03 PM, "dev on behalf of Brent Putman" [hidden email] wrote:

Although I didn't implement this originally (Chad did),
The part I thought you did was the "load initially from backup and then do a refresh", which I had the impression was a recent change we made in response to the performance problems. That seems to be the behavior under discussion as you alluded to later.

Oh, yes, I did implement that recent performance change.  I didn't implement the original provider code, which still had the file-backed aspect.  It's not swapped in to my brain, but I think it used to only load from the backup file if the HTTP fetch failed?  I might have that completely wrong.   But to me that's not fundamentally different - I think Tom would say that it shouldn't load it period if it's not signed, and that's what I thought we were discussing.

And the current behavior has the same practical result as the former behavior, just with a few seconds delay: unless there's a fetch or validation problem, you're going to ultimately get a freshly fetched set of data. And if there is a problem, you wind up with the backup data.

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Tom Scavo
Thanks all. I appreciate your patience and feedback. Since there's no
support for my (evidently whacky) idea, I should just drop this.
Before I do, I'll pose one further question along these lines:

I'm not clear why there are two HTTPMetadataProviders. If the only
difference between the two is the backing file, you only need one
MetadataProvider, right? Is there some historical basis for having
two? If you have any insight, please let me know.

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Cantor, Scott E.
> I'm not clear why there are two HTTPMetadataProviders. If the only
> difference between the two is the backing file, you only need one
> MetadataProvider, right? Is there some historical basis for having
> two? If you have any insight, please let me know.

I alluded to that earlier. Other than code factoring I don't know why two types are needed for any normal deployment. I suppose there could be embedded scenarios where that code might be useful to somebody that couldn't back the file up.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Tom Scavo
On Thu, Dec 21, 2017 at 9:13 AM, Cantor, Scott <[hidden email]> wrote:
>> I'm not clear why there are two HTTPMetadataProviders. If the only
>> difference between the two is the backing file, you only need one
>> MetadataProvider, right? Is there some historical basis for having
>> two? If you have any insight, please let me know.
>
> I alluded to that earlier. Other than code factoring I don't know why two types are needed for any normal deployment. I suppose there could be embedded scenarios where that code might be useful to somebody that couldn't back the file up.

Sure, but in any case, why are two HTTPMetadataProviders needed? Why
couldn't there be just one, with an optional backingFile attribute?

I'll create a jira issue along these lines tomorrow unless someone
suggests otherwise.

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Cantor, Scott E.
On 1/4/18, 10:50 AM, "dev on behalf of Tom Scavo" <[hidden email] on behalf of [hidden email]> wrote:

> Sure, but in any case, why are two HTTPMetadataProviders needed? Why
> couldn't there be just one, with an optional backingFile attribute?

Probably but after finally (it seems) getting this code stable after 15 years, I don't see much value is messing with it simply to "unexpose" an unused class.

-- Scott


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: include SignatureValidation filter with FileBackedHTTPMetadataProvider

Tom Scavo
On Thu, Jan 4, 2018 at 11:11 AM, Cantor, Scott <[hidden email]> wrote:
> On 1/4/18, 10:50 AM, "dev on behalf of Tom Scavo" <[hidden email] on behalf of [hidden email]> wrote:
>
>> Sure, but in any case, why are two HTTPMetadataProviders needed? Why
>> couldn't there be just one, with an optional backingFile attribute?
>
> Probably but after finally (it seems) getting this code stable after 15 years, I don't see much value is messing with it simply to "unexpose" an unused class.

Yes, that's a choice I suppose. In any case, I filed an issue:
https://issues.shibboleth.net/jira/browse/IDP-1243

Thanks,

Tom
--
To unsubscribe from this list send an email to [hidden email]