Live tests fail for v6.14 on Windows 10 Strawberry Perl 5.24.1.1 behind HPE firewall

[pwr22]

Installing 6.13 works fine. I am behind a restrictive outbound firewall with a proxy for external access, http_proxy and https_proxy are set correctly

Perl -V

Summary of my perl5 (revision 5 version 24 subversion 1) configuration:

  Platform:
    osname=MSWin32, osvers=6.3, archname=MSWin32-x64-multi-thread
    uname='Win32 strawberry-perl 5.24.1.1 #1 Mon Jan 16 02:00:29 2017 x64'
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags =' -s -O2 -DWIN32 -DWIN64 -DCONSERVATIVE  -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -fwrapv -fno-strict-aliasing -mms-bitfields',
    optimize='-s -O2',
    cppflags='-DWIN32'
    ccversion='', gccversion='4.9.2', gccosandvers=''
    intsize=4, longsize=4, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='long long', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='g++', ldflags ='-s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib"'
    libpth=C:\STRAWB~1\c\lib C:\STRAWB~1\c\x86_64-w64-mingw32\lib C:\STRAWB~1\c\lib\gcc\x86_64-w64-mingw32\4.9.2
    libs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    perllibs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    libc=, so=dll, useshrplib=true, libperl=libperl524.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=xs.dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-mdll -s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib"'


Characteristics of this binary (from libperl):
  Compile-time options: HAS_TIMES HAVE_INTERP_INTERN MULTIPLICITY
                        PERLIO_LAYERS PERL_COPY_ON_WRITE
                        PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
                        PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS
                        PERL_MALLOC_WRAP PERL_PRESERVE_IVUV USE_64_BIT_INT
                        USE_ITHREADS USE_LARGE_FILES USE_LOCALE
                        USE_LOCALE_COLLATE USE_LOCALE_CTYPE
                        USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
                        USE_PERL_ATOF
  Built under MSWin32
  Compiled at Jan 16 2017 02:12:13
  @INC:
    C:/Strawberry/perl/site/lib/MSWin32-x64-multi-thread
    C:/Strawberry/perl/site/lib
    C:/Strawberry/perl/vendor/lib
    C:/Strawberry/perl/lib
    .

cpanm Net::HTTP@6.14

cpanm (App::cpanminus) 1.7043 on perl 5.024001 built for MSWin32-x64-multi-thread
Work directory is C:\Users\rpeter/.cpanm/work/1493136176.12116
You have make C:\Strawberry\c\bin\dmake.exe
You have LWP 6.26
Falling back to Archive::Tar 2.24
Searching Net::HTTP (== 6.14) on cpanmetadb ...
--> Working on Net::HTTP
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/Net-HTTP-6.14.tar.gz
-> OK
Unpacking Net-HTTP-6.14.tar.gz
Entering Net-HTTP-6.14
Checking configure dependencies from META.json
Checking if you have ExtUtils::MakeMaker 6.58 ... Yes (7.24)
Configuring Net-HTTP-6.14
Running Makefile.PL
Checking if your kit is complete...
Looks good
Generating a dmake-style Makefile
Writing Makefile for Net::HTTP
Writing MYMETA.yml and MYMETA.json
-> OK
Checking dependencies from MYMETA.json ...
Checking if you have IO::Socket::INET 0 ... Yes (1.35)
Checking if you have IO::Uncompress::Gunzip 0 ... Yes (2.070)
Checking if you have warnings 0 ... Yes (1.36)
Checking if you have Data::Dumper 0 ... Yes (2.161)
Checking if you have Test::More 0 ... Yes (1.302083)
Checking if you have strict 0 ... Yes (1.11)
Checking if you have File::Spec 0 ... Yes (3.63_01)
Checking if you have base 0 ... Yes (2.23)
Checking if you have Carp 0 ... Yes (1.40)
Checking if you have Socket 0 ... Yes (2.024)
Checking if you have Compress::Raw::Zlib 0 ... Yes (2.071)
Checking if you have IO::Select 0 ... Yes (1.22)
Checking if you have ExtUtils::MakeMaker 0 ... Yes (7.24)
Checking if you have URI 0 ... Yes (1.71)
Checking if you have vars 0 ... Yes (1.03)
Building and testing Net-HTTP-6.14
cp lib/Net/HTTP/NB.pm blib\lib\Net\HTTP\NB.pm
cp lib/Net/HTTPS.pm blib\lib\Net\HTTPS.pm
cp lib/Net/HTTP.pm blib\lib\Net\HTTP.pm
cp lib/Net/HTTP/Methods.pm blib\lib\Net\HTTP\Methods.pm
"C:\Strawberry\perl\bin\perl.exe" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib\lib', 'blib\arch')" t/*.t
# 
# Versions for all modules listed in MYMETA.json (including optional ones):
# 
# === Configure Requires ===
# 
#     Module              Want Have
#     ------------------- ---- ----
#     ExtUtils::MakeMaker  any 7.24
# 
# === Configure Suggests ===
# 
#     Module      Want    Have
#     -------- ------- -------
#     JSON::PP 2.27300 2.27400
# 
# === Build Requires ===
# 
#     Module              Want Have
#     ------------------- ---- ----
#     ExtUtils::MakeMaker  any 7.24
# 
# === Test Requires ===
# 
#     Module              Want     Have
#     ------------------- ---- --------
#     Data::Dumper         any    2.161
#     ExtUtils::MakeMaker  any     7.24
#     File::Spec           any  3.63_01
#     IO::Select           any     1.22
#     Socket               any    2.024
#     Test::More           any 1.302083
# 
# === Test Recommends ===
# 
#     Module         Want     Have
#     ---------- -------- --------
#     CPAN::Meta 2.120900 2.150010
# 
# === Runtime Requires ===
# 
#     Module                 Want  Have
#     ---------------------- ---- -----
#     Carp                    any  1.40
#     Compress::Raw::Zlib     any 2.071
#     IO::Socket::INET        any  1.35
#     IO::Uncompress::Gunzip  any 2.070
#     URI                     any  1.71
#     base                    any  2.23
#     strict                  any  1.11
#     vars                    any  1.03
#     warnings                any  1.36
# 
# === Runtime Suggests ===
# 
#     Module            Want  Have
#     ----------------- ---- -----
#     IO::Socket         any  1.38
#     IO::Socket::INET6  any  2.72
#     IO::Socket::IP     any  0.38
#     IO::Socket::SSL   1.38 2.043
#     Symbol             any  1.07
# 
t/00-report-prereqs.t .. ok
t/http-nb.t ............ skipped: This test doesn't work on Windows
t/http.t ............... ok
Bad file descriptor at t/live-https.t line 34.
# Looks like your test exited with 9 before it could output anything.
t/live-https.t ......... 
Dubious, test returned 9 (wstat 2304, 0x900)
Failed 6/6 subtests 
Server closed connection without sending any data back at C:\Users\rpeter\.cpanm\work\1493136176.12116\Net-HTTP-6.14\blib\lib/Net/HTTP/Methods.pm line 391.
# Looks like your test exited with 255 just after 3.
t/live.t ............... 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 3/6 subtests 

#   Failed test 'base response length is less than 1024: 11847'
#   at t/rt-112313.t line 84.
Negative repeat count does nothing at t/rt-112313.t line 50.
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
# Looks like you failed 19 tests of 40.
t/rt-112313.t .......... 
Dubious, test returned 19 (wstat 4864, 0x1300)
Failed 19/40 subtests 

Test Summary Report
-------------------
t/live-https.t       (Wstat: 2304 Tests: 0 Failed: 0)
  Non-zero exit status: 9
  Parse errors: Bad plan.  You planned 6 tests but ran 0.
t/live.t             (Wstat: 65280 Tests: 3 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 6 tests but ran 3.
t/rt-112313.t        (Wstat: 4864 Tests: 40 Failed: 19)
  Failed tests:  1, 4, 6, 8, 10, 12, 14, 17, 19, 21, 23
                25, 27, 30, 32, 34, 36, 38, 40
  Non-zero exit status: 19
Files=6, Tests=81, 26 wallclock secs ( 0.02 usr +  0.02 sys =  0.03 CPU)
Result: FAIL
Failed 3/6 test programs. 19/81 subtests failed.
dmake.exe:  Error code 147, while making 'test_dynamic'
-> FAIL Installing Net::HTTP failed. See C:\Users\rpeter\.cpanm\work\1493136176.12116\build.log for details. Retry with --force to force install it.

[genio]

Thanks for the report. I'll do some testing when I get home with access to a Windows machine.

[oalders]

@skaji possibly related to your changes?

[skaji]

First I want to make sure whether tests pass or not with Net::HTTP@6.13.

@pwr22 Could you test this please?

git clone https://github.com/libwww-perl/Net-HTTP
cd Net-HTTP
git checkout v6.13
touch LIVE_TESTS
perl Makefile.PL
make
make test

[update]

• If you don't have touch command, replace it by perl -E 'say 1' > LIVE_TESTS.
• If you don't have make command but dmake, then replace make by dmake.

[genio]

Or, cpanm --test-only Net::HTTP@6.13

[skaji]

We should create LIVE_TESTS file manually to run tests.
So cpanm --test-only Net::HTTP@6.13 does not run tests :)

[genio]

I see. Sorry, I completely missed that point.

[skaji]

As far as I know, Net::HTTP itself does not respect https?_proxy environment variable (LWP does).

@pwr22
I think you cannot access www.cpan.org or httpbin.org without proxy, so three tests (t/live-https.t, t/live.t, t/rt-112313.t) fail.

[pwr22]

@skaji enabling the live tests on 6.13 causes failures also. The difference being that they aren't run by default when installing that version though

I don't think it's a good idea to run live tests by default on build unless you have some way of doing a sanity check on connectivity first. Given that this is a HTTP module itself.... that's a bit of a bootstrapping problem

At work I'm stuck behind a firewall that stops pretty much all external connectivity. I'm 95% sure that's the problem here too

[pwr22]

Double checked on Test::RequiresInternet. Looks like it has no non-core deps so maybe the live tests could make use of it?

[skaji]

@pwr22 Thank you for checking.

@oalders @genio
I think we have two options to fix tests:

• (A) Use Test::RequiresInternet to check test network connectivity.
• (B) Use an opts-in environment variable (such as TEST_LIVE or LIVE_TEST). If it is true, then run tests. Otherwise, skip them.

[oalders]

I'd be inclined to try (A) and see how that works out.

[pwr22]

@oalders @skaji I'm happy to put together a PR for (A) if you'd like?

[skaji]

@pwr22 Yes! thanks!

[pwr22]

Hmm, I don't think (A) will be a viable fix for my environment. The firewall seems to respond in such a way that both the current tests sanity check (which I didn't know existed - sorry for missing that!) and Test::RequiresInternet don't catch it

[skaji]

@pwr22 Could you show us the outputs for the following commands?

perl -E "use Test::RequiresInternet q(www.cpan.org) =>  80; say q(pass)"
perl -E "use Test::RequiresInternet q(www.cpan.org) => 443; say q(pass)"
perl -E "use Test::RequiresInternet q(httpbin.org)  =>  80; say q(pass)"
perl -MLWP::UserAgent -E "print LWP::UserAgent->new(env_proxy => 0)->get(q(http://www.cpan.org))->dump"
perl -MLWP::UserAgent -E "print LWP::UserAgent->new(env_proxy => 1)->get(q(http://www.cpan.org))->dump"