mod_shib_24 fails to load after upgrade

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

mod_shib_24 fails to load after upgrade

Jason Howe
Hi All,

We upgraded all our webservers to SP 3 today.  On 2 out of about 15
machines, httpd failed to restart after the upgrade with the following
error:
httpd: Syntax error on line 96 of /etc/httpd/conf/httpd.conf: Syntax
error on line 13 of /etc/httpd/conf.d/shib.conf: Cannot load
/usr/lib64/shibboleth/mod_shib_24.so into server:
/usr/lib64/libxmltooling-lite.so.8: undefined symbol: deflateInit2_

I've verified that /usr/lib64/libxmltooling-lite.so.8 is in place, and
the md5sum matches to that of a working machine.   Running ldconfig
didn't change anything.

I downgraded back to shibboleth-2.6.1-3.1.x86_64
opensaml-schemas-2.6.1-3.1.x86_64 xmltooling-schemas-1.6.4-3.1.x86_64
and things came back up.

These are standard Centos7.5 machines running stock Apache 2.4.  Any
suggestions?  I'm kind of at a loss right now.

Thanks,
Jason



--
--
Jason Howe
Software Engineer, Web Applications
Paul Allen School for Computer Science & Engineering
University of Washington

--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: mod_shib_24 fails to load after upgrade

Cantor, Scott E.
On 7/19/18, 8:36 PM, "users on behalf of Jason Howe" <[hidden email] on behalf of [hidden email]> wrote:

> I've verified that /usr/lib64/libxmltooling-lite.so.8 is in place, and
> the md5sum matches to that of a working machine.   Running ldconfig
> didn't change anything.

It's not finding zlib, which would suggest the build didn't pull it in as an RPM requirement so the working machines have it and the non-working ones don't. I don't have enough understanding of RPM to know why it wouldn't have identified it as a runtime requirement but I know enough to run the queries and verify what it did so I can verify what it thinks it needs.

> These are standard Centos7.5 machines running stock Apache 2.4.  Any
> suggestions?  I'm kind of at a loss right now.

If zlib isn't there, I assume installing zlib would address it, because there's no way it could work on any box otherwise and it worked fine on mine.

-- Scott


--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: mod_shib_24 fails to load after upgrade

Cantor, Scott E.
Actually, that can't be it, because libsaml used to be linked to zlib, so it has to have been there from the older version. It's the same dependency, it just moved down a library from one to the other. I have no idea what it could be but I'll still check and see what the rpm commands say.

-- Scott

On 7/19/18, 9:08 PM, "Cantor, Scott" <[hidden email]> wrote:

On 7/19/18, 8:36 PM, "users on behalf of Jason Howe" <[hidden email] on behalf of [hidden email]> wrote:

> I've verified that /usr/lib64/libxmltooling-lite.so.8 is in place, and
> the md5sum matches to that of a working machine.   Running ldconfig
> didn't change anything.

It's not finding zlib, which would suggest the build didn't pull it in as an RPM requirement so the working machines have it and the non-working ones don't. I don't have enough understanding of RPM to know why it wouldn't have identified it as a runtime requirement but I know enough to run the queries and verify what it did so I can verify what it thinks it needs.

> These are standard Centos7.5 machines running stock Apache 2.4.  Any
> suggestions?  I'm kind of at a loss right now.

If zlib isn't there, I assume installing zlib would address it, because there's no way it could work on any box otherwise and it worked fine on mine.

-- Scott



--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: mod_shib_24 fails to load after upgrade

Cantor, Scott E.
And moreover, no, mod_shib should not be linked to the zlib-requiring code, it's linked to libxmltooling-lite, which does not build against zlib. So there's something deeply wrong, and my guess is the working boxes have an Apache module loaded that happen to get zlib into the process and the non-working ones don't.

I'll have to investigate the build more fully but that's about all I can say for now without a (non)-working example and time to double check it all. Nothing is jumping out at me as wrong with the makefile, but I suspect there's a stray bit of code somewhere that must have crept in that shouldn't be in that side of the build. If I can find it, it's just a quick patch but we'll see.

-- Scott


--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: mod_shib_24 fails to load after upgrade

Cantor, Scott E.
I found the offending code, but the expedient patch will be to just link that half to libz and leave the code alone, it's not hurting anything and it's a smaller change. I'll bump the library again just to limit confusion. I should be able to get that pushed out tonight and nothing else should need a rebuild.

-- Scott


--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: mod_shib_24 fails to load after upgrade

Cantor, Scott E.
The updated package is being copied to the external mirror now, it's already available from the primary site. I believe that should fix the problem, thanks for finding that.

-- Scott


--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: mod_shib_24 fails to load after upgrade

Jason Howe
Wow, thanks for amazing response on this.  I'll test it tomorrow as soon
as it shows up in our local mirror and report back.

--Jason



On 07/19/2018 07:18 PM, Cantor, Scott wrote:
> The updated package is being copied to the external mirror now, it's already available from the primary site. I believe that should fix the problem, thanks for finding that.
>
> -- Scott
>
>

--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: mod_shib_24 fails to load after upgrade

Jason Howe
In reply to this post by Cantor, Scott E.
Happy to report that the update fixed my issue. 

I did a little more digging, to see what apache module differences there
might be between the working and non working systems, sorry to say that
once of the non-working system's apache module list was identical to
that of a working system.  One had a slight divergence with the
inclusion of mod_dav_svn.  There was no common divergence between the
non-working and working systems.

Regardless, thanks again for the fix!

--Jason


On 07/19/2018 07:18 PM, Cantor, Scott wrote:
> The updated package is being copied to the external mirror now, it's already available from the primary site. I believe that should fix the problem, thanks for finding that.
>
> -- Scott
>
>

--
--
Jason Howe
Software Engineer, Web Applications
Paul Allen School for Computer Science & Engineering
University of Washington

--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: mod_shib_24 fails to load after upgrade

Peter Schober
* Jason Howe <[hidden email]> [2018-07-20 18:48]:
> I did a little more digging, to see what apache module differences there
> might be between the working and non working systems

You'd need to look not just at Apache httpd modules, but all RPM
packages. E.g. comparing the output of `rpm -qa` on the systems, AFAIU.
-peter
--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: mod_shib_24 fails to load after upgrade

Cantor, Scott E.
In reply to this post by Jason Howe
On 7/20/18, 12:48 PM, "Jason Howe" <[hidden email]> wrote:

> Happy to report that the update fixed my issue.

Thank you for confirming.

> There was no common divergence between the
> non-working and working systems.

Hmm, something clearly was or wasn't linked to libz but it's not worth worrying about since it was a build mistake anyway.

-- Scott


--
For Consortium Member technical support, see https://wiki.shibboleth.net/confluence/x/coFAAg
To unsubscribe from this list send an email to [hidden email]