The length of the prime was used incorrectly and this resulted
in WPS DH operation failing whenever the public key ended up having
leading zeros (i.e., about every 1/256th time).
Mainly, this is including header files to get definitions for functions
which is good to verify that the parameters match. None of these are
issues that would have shown as incorrect behavior of the program.
OpenSSL 0.9.7 does not include get_rfc3526_prime_1536() function, so
provide that functionality internally if needed. In addition, make
sha256_vector() building depend on whether SHA256 support is included
in the OpenSSL library. This with CONFIG_INTERNAL_SHA256=y in .config
allows OpenSSL without SHA256 support to be used.
Crypto library wrappers can now override the internal DH (group 5)
implementation. As a starting point, this is done with OpenSSL. The
new mechanism is currently available only for WPS (i.e., IKEv2 still
depends on the internal DH implementation).
Better not specify EVP_CIPHER again for the second init call since that
will override key length with the default value. The previous version
was likely to work since most use cases would be likely to use the
default key length. Anyway, better make this handle variable length
ciphers (mainly, RC4), too, just in case it is needed in the future.
Instead of using low level, digest-specific functions, use the generic
EVP interface for digest functions. In addition, report OpenSSL errors
in more detail.
Some crypto libraries can return in these functions (e.g., if a specific
hash function is disabled), so we better provide the caller a chance to
check whether the call failed. The return values are not yet used
anywhere, but they will be needed for future changes.
