MAINT Skip unstable GPR tests on 32 bit Python #20334

ogrisel · 2021-06-23T08:13:41Z

Those tests failed with Python 3.7 and 3.8 on 32 bit linux on the nightly [cd build] runs since a few days (possibly related to scipy 1.7.0) but those tests were also previously skipped on Python 3.6.

Lists of impacted tests:

test_gpr_interpolation
test_lml_improving
test_predict_cov_vs_std

Observed failures:

      @pytest.mark.parametrize("kernel", kernels)
      def test_gpr_interpolation(kernel):
          if sys.maxsize <= 2 ** 32 and sys.version_info[:2] == (3, 6):
              pytest.xfail("This test may fail on 32bit Py3.6")
      
          # Test the interpolating property for different kernels.
          gpr = GaussianProcessRegressor(kernel=kernel).fit(X, y)
          y_pred, y_cov = gpr.predict(X, return_cov=True)
      
  >       assert_almost_equal(y_pred, y)
  E       AssertionError: 
  E       Arrays are not almost equal to 7 decimals
  E       
  E       Mismatched elements: 6 / 6 (100%)
  E       Max absolute difference: 3.21747557
  E       Max relative difference: 7.59985682
  E        x: array([ 1.9365826, -2.7941155, -2.407411 , -0.29458  ,  3.0978149,
  E               7.7699864])
  E        y: array([ 0.841471 ,  0.42336  , -4.7946214, -1.676493 ,  4.5989062,
  E               7.914866 ])

      @pytest.mark.parametrize("kernel", non_fixed_kernels)
      def test_lml_improving(kernel):
          if sys.maxsize <= 2 ** 32 and sys.version_info[:2] == (3, 6):
              pytest.xfail("This test may fail on 32bit Py3.6")
      
          # Test that hyperparameter-tuning improves log-marginal likelihood.
          gpr = GaussianProcessRegressor(kernel=kernel).fit(X, y)
  >       assert gpr.log_marginal_likelihood(gpr.kernel_.theta) > gpr.log_marginal_likelihood(
              kernel.theta
          )
  E       assert -111266738064.26326 > -48.88011095337428

Note that test_predict_cov_vs_std was not recently failing but I suspect that this is not numerically stable on 32 bit Python either because it used to fail under Python 3.6.

ogrisel · 2021-06-23T08:47:56Z

There is another unrelated error when building scikit-learn on win32 for Python 3.7:

    Partial import of sklearn during the build process.
    error: Command "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30037\bin\HostX86\x86\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:c:\cibw\python\pythonx86.3.7.9\tools\libs /LIBPATH:c:\cibw\python\pythonx86.3.7.9\tools\PCbuild\win32 /LIBPATH:build\temp.win32-3.7 /LIBPATH:"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30037\ATLMFC\lib\x86" /LIBPATH:"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30037\lib\x86" /LIBPATH:"C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x86" /LIBPATH:"C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x86" /LIBPATH:"C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x86" /LIBPATH:"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30037\ATLMFC\lib\x86" /LIBPATH:"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30037\lib\x86" /LIBPATH:"C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x86" /LIBPATH:"C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x86" /LIBPATH:"C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x86" /LIBPATH:"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30037\ATLMFC\lib\x86" /LIBPATH:"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30037\lib\x86" /LIBPATH:"C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x86" /LIBPATH:"C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x86" /LIBPATH:"C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x86" /EXPORT:PyInit__random build\temp.win32-3.7\Release\sklearn\utils\_random.obj /OUT:build\lib.win32-3.7\sklearn\utils\_random.cp37-win32.pyd /IMPLIB:build\temp.win32-3.7\Release\sklearn\utils\_random.cp37-win32.lib /openmp" failed with exit status 1158
    Building wheel for scikit-learn (PEP 517): finished with status 'error'

It seems to be related the configuration of the Windows linker for this build.

Should probably be tackled in a dedicated PR because I am not sure about the proper fix and don't want to make this PR more complicated.

ogrisel · 2021-06-23T08:49:00Z

The fetch_openml/numpy.unique deprecation warning was solved concurrently and merged in main.

glemaitre

LGTM regarding the GPR side. It seems to be problematic but not linked to this PR because the results show that the maximization of the log-likelihood does not work on 32 bits system.

ogrisel · 2021-06-23T13:01:48Z

I opened #20337 to track the 32 bit numerical stability issues in GPR.

MAINT Skip unstable GPR tests on 32 bit Python [cd build]

6ff2c27

github-actions bot added the module:gaussian_process label Jun 23, 2021

ogrisel added No Changelog Needed Build / CI module:test-suite labels Jun 23, 2021

glemaitre approved these changes Jun 23, 2021

View changes

ogrisel mentioned this pull request Jun 23, 2021

Numerical stability problems in GaussianProcessRegressor on 32 bit Python #20337

Open

ogrisel deleted the ogrisel:skip-unstable-gpr-tests-on-32-bit-python branch Jun 23, 2021

scikit-learn / scikit-learn

MAINT Skip unstable GPR tests on 32 bit Python #20334

MAINT Skip unstable GPR tests on 32 bit Python #20334

ogrisel commented Jun 23, 2021 •

edited

ogrisel commented Jun 23, 2021 •

edited

ogrisel commented Jun 23, 2021

glemaitre left a comment

ogrisel commented Jun 23, 2021

scikit-learn / scikit-learn

Sponsor scikit-learn/scikit-learn

MAINT Skip unstable GPR tests on 32 bit Python #20334

MAINT Skip unstable GPR tests on 32 bit Python #20334

Conversation

ogrisel commented Jun 23, 2021 • edited

ogrisel commented Jun 23, 2021 • edited

ogrisel commented Jun 23, 2021

glemaitre left a comment

ogrisel commented Jun 23, 2021

ogrisel commented Jun 23, 2021 •

edited

ogrisel commented Jun 23, 2021 •

edited