SubVersion modules for x64

Started by Sob, August 31, 2010, 04:18:14 AM

Previous topic - Next topic

Sob

Those __imp__XML_* symbols are from Expat. Pointing the linker to libexpat.lib/libexpatw.lib (I'm not really sure which one) should help.

Then it leaves only _SSL_SESSION_cmp. It's from OpenSSL and it seems that it has been removed in 1.0.0. Quick research reveals that Neon can live with it (see lines 637-648 in ne_openssl.c). So it's probably just problem with defines, HAVE_SSL_SESSION_CMP and/or SSL_SESSION_cmp incorrectly defined somewhere?

mario

HAVE_SSL_SESSION_CMP is nowhere defined in the source code.

SSL_SESSION_cmp is used / defined

openssl-1.0.0d\ssl\ssl_lib.c line arround 438,1498
openssl-1.0.0d\util\ssleay.num line 120
openssl-1.0.0d\doc\ssl\ssl.pod line 395

subversion-1.6.17\neon\src\ne_openssl.c line 678



Sob

I wanted to try it myself, this time the proper way, full build with everything as you do. Can't say it went too well.

First there was problem with Apache 2.2.19. Release/x64 config had more than half of subprojects (modules, etc..) set to Win32 (x86), so they didn't build. I didn't find an easy way to add x64 target only where it was missing, so I had to remove x64 completely and then recreate it from Win32. Apache then built fine, but it messed up output paths (added x64 subdirectory). But ok..

With SubVersion I added all required dependencies, but then I still hit the exactly same problem as before, missing svn's own symbols everywhere. Dlls (libsvn*.dll) compiled just fine, but the symbols required by exes and Apache modules were not in libsvn*.lib where I'd expect them, but only in svn*.lib. And linking to them produces static executables and modules that don't need svn dlls. I have no idea what I've done wrong. Maybe it's somehow related to Apache build problems, gen-make.py looking for something it couldn't find and creating incomplete project files? But it didn't complain about anything.

I had no missing _SSL_SESSION_cmp in libsvn_ra_dll project, but since it all went differently than your build, I'm not sure that it says anything useful.

The only idea I have for you is to try removing the conditions around Neon-defined SSL_SESSION_cmp, to be sure that it's always defined.

Gregg

Sob,

cvtdsp64.bat
@rem = '--*-Perl-*--
@echo off
if "%OS%" == "Windows_NT" goto WinNT
perl -x -S "%0" %1 %2 %3 %4 %5 %6 %7 %8 %9
goto endofperl
:WinNT
perl -x -S %0 %*
if NOT "%COMSPEC%" == "%SystemRoot%\system32\cmd.exe" goto endofperl
if %errorlevel% == 9009 echo You do not have Perl in your PATH.
if errorlevel 1 goto script_failed_so_exit_with_non_zero_val 2>nul
goto endofperl
@rem ';
#!C:\usr\bin\perl.EXE -w
#line 15
##################################################################
# Script Name: Convert Apache DSP files to x64
# FileName: cvtdsp64.pl
# Date: November 18, 2010
#
# revision 2010-1212-1552 [Version 1.0]
#
##################################################################
my $osh = ".\\os\\win32\\os.h";
my @LISTDSP = ('DIR','/B','/S','*.DSP','>','DSPS.list');
my @MODSMK = ('DIR','/B','/S','modules.mk.win','>>','DSPS.list');
system(@LISTDSP);
system(@MODSMK);
my @dsps = read_file('./DSPS.list');
my @OSH = read_file($osh);
foreach (@dsps) {
  my @dspcontents = read_file($_);
  print "Converting ".$_."\n";
  $prepped =  join("\n",@dspcontents);

  my $pcre = "TRUE" if $_ =~ /srclib\\pcre/i && $prepped =~ /\/x64/;

  if ($prepped =~ /machine:(X86|I386)/i) {
    $prepped =~ s/machine:(X86|I386)/machine:X64/gi; # for it exists, change it
  }
  else {
    if ($_ =~ /(support|httpd\.dsp)/i && $prepped !~ /machine:X64/i) {
      $prepped =~ s/subsystem:console/subsystem:console \/machine:X64/gi; # add for dll & so w/ opt:ref
    }
    if ($_ =~ /(libhttpd|ApacheMonitor)\.dsp/i && $prepped !~ /machine:X64/i) {
      $prepped =~ s/subsystem:windows/subsystem:windows \/machine:X64/gi; # add for dll & so w/ opt:ref
    }
    $prepped=~ s/\/debug \/out:/\/debug \/machine:X64 \/out:/gi;
  }

  $prepped=~ s/baseaddr\.ref/baseaddr64\.ref/gi; # for x64 baseaddr
  $prepped=~ s/\/mktyplib203 \/win32/\/mktyplib203 \/x64/gi; # for x64 midl
  $prepped=~ s/\/mktyplib203 \/o \/win32/\/mktyplib203 \/o \/x64/gi; # for x64 midl
  $prepped=~s/\/D "WIN32"/\/D "WIN32" \/D "WIN64"/gi; # unlock x64 specific code
  $prepped=~s/\/MACHINE:(X86|I386)/\/machine:X64/gi; # catch any /machine that slid by
  $prepped=~s/x64\\Debug/Debug/gi; # for putting x64 dlls in x86 locations Debug
  $prepped=~s/x64\\Release/Release/gi; # for putting x64 dlls in x86 locations Release
  $prepped=~s/x64\\LibD/LibD/gi; #for putting x64 libs in x86 locations LibR
  $prepped=~s/x64\\LibR/LibR/gi; #for putting x64 libs in x86 locations LibR
  $prepped=~ s/LIB32 \/nologo \/out:/LIB32 \/nologo \/machine:X64 \/out:/g; # for static libs
  unless ($_ =~ /srclib\\apr/i || $pcre) {
    #
    # Already have x64 targets in APR and the pcre projects for Apache 2.3 AH,
    # so do not modify the Win32 targets in APR
    #
    $prepped=~ s/Win32 Release/x64 Release/gi ; # convert Win32 Release targets to x64
    $prepped=~ s/Win32 Debug/x64 Debug/gi ; # convert Win32 Debug targets to x64
  }
  skipit:
  open(WRI,">".$_) || die ("Can't open $_ for output");
  print WRI $prepped."\n";
  close(WRI);

}

$oshprep = join("\n",@OSH);
unless ($oshprep =~ /\#define PLATFORM "Win64"/) {
  $oshprep =~ s/^\#define PLATFORM "Win32"/\#define PLATFORM "Win64"\n\#define WIN64/g;
  open(WRI,">".$osh) || die ("Can't open $osh for output");
  print WRI $oshprep."\n";
  close(WRI);
}

my @CLEANUP = ('DEL','/F','/Q','DSPS.list');
system(@CLEANUP);

exit 0;
# END main()

# Read File ######################################################
sub read_file {
  (my $read_file)=@_;
  my @filecontents;
  $read_file=~s/\n//;
  if (-e "$read_file") {
    open(READ,"$read_file") || return ("Can't Open File: $read_file\n");
    @filecontents = <READ>;
    close(READ);
  }
  else {
    return ("Cannot find file: $read_file\n");
  }
  chomp @filecontents;
  return @filecontents;
} # End sub read_file

##################################################################
#  End cvtdsp64.bat                                                #

__END__
:endofperl
@rem ';




mario

Here my short way it worked to compile subversion /until now ;)

Get
http://subversion.tigris.org/downloads/subversion-1.6.17.zip and http://subversion.tigris.org/downloads/subversion-deps-1.6.17.zip

C:\python27\python gen-make.py -t vcproj --vsnet-version=2008 --with-httpd=C:\build\httpd-2.2.19-sni6-x64
--with-openssl=C:\build\httpd-2.2.19-sni6-x64\srclib\openssl
--with-zlib=C:\build\subversion-1.6.17-x64\zlib --with-apr=C:\build\subversion-1.6.17-x64\apr

Open generated build\win32\build_zlib.bat and insert at line 51 (before "set BUILD_OPTS=..." line):
set ASM_OPTS=AS=ml64 LOC="-DASMV -DASMINF -I." OBJA="inffasx64.obj gvmat64.obj inffas8664.obj"

OPEN IDE



x86

C:\python27\python gen-make.py -t vcproj --vsnet-version=2008 --with-httpd=C:\build\httpd-2.2.19-sni6-x86
--with-openssl=C:\build\httpd-2.2.19-sni6-x86\srclib\openssl
--with-zlib=C:\build\subversion-1.6.17-x86\zlib --with-apr=C:\build\subversion-1.6.17-x86\apr

Open generated build\win32\build_zlib.bat and insert at line 51 (before "set BUILD_OPTS=..." line):
set ASM_OPTS=LOC="-DASMV -DASMINF -I." OBJA="inffas32.obj match686.obj"

OPEN IDE

Sob

Ok, I tried it again:

  • cvtdsp64.bat helped, but the resulting project files had problems when compiling resources (rc.exe call)
  • srclib\apr\build\cvtdsp.pl -2005 fixed that
  • there were complaints (errors) about missing BaseAddr64.ref
  • not sure about it, I copied it from BaseAddr.ref and it changed errors to warnings
  • good enough for Apache to compile ;)
  • with SubVersion my original linking problem was caused by Python not in PATH; I only had .py associated, but build process calls it directly and I overlooked that before
  • neon 0.28.3 from deps does not have workaround for SSL_SESSION_cmp removed from OpenSSL, so it fails with unresolved symbol
  • neon 0.29.6 has the workaround (own function) and problem is gone
  • missing Expat symbols (__imp__XML_*) remain
  • I first thought about linking to external Expat (if it wants it, just let it have it), but something felt wrong about it; if problem was only with linking, it must have got the headers from somewhere and .lib should be from the same source
  • that source is httpd\srclib\apr-util\xml\expat\lib and lib is xml.lib, everything seemed to be prepared, yet it didn't work
  • the missing symbols had extra "__imp__" prefixed and it came from #define XMLIMPORT __declspec(dllimport) in httpd\srclib\apr-util\xml\expat\lib\expat.h and it was conditional
  • jump right to the end, the magic solution is to add "/D XML_STATIC" to EXPAT_FLAGS in neon\neon.mak, line 57
  • build ok :)

Gregg

well .. perl srclib\apr\build\ctvdsp.pl -2005 still needs to be done .. after the x64 convert .. should of mentioned it ... for me it is automatic .. I don't even think about it when building .. my body is trained to just do without thinking .. which sometimes gets me in trouble  ;)

Sob

And what about that missing BaseAddr64.ref? Original source has BaseAddr.ref and the file exists. Running cvtdsp64.bat changes dependencies to BaseAddr64.ref, but does not provide the file in any way.

Building of many components then ends up with:
LINK : fatal error LNK1104: cannot open file 'os\win32\baseaddr64.ref'

After providing own BaseAddr64.ref, either the copy of BaseAddr.ref or just empty file, the result is:
os\win32\baseaddr64.ref : warning LNK4198: base key 'libhttpd.dll /opt:ref' not found - using default

I asked Google, but got only three results for BaseAddr64.ref, all from this thread. Not very helpful. ;D

Gregg

Yes, it's not perfect, and you get that. I am going to just remove the /base@ altogether. It's on my list of todo's. just /opt:ref should be fine.

Those are hardcoded in apr, I was thinking of giving it an attempt again. There is a magic number that needs to be added to the addresses in the 32 bit one to be inline with x64. I cannot remember but have it written down somewhere.

mario

upgrading neon and adding "/D XML_STATIC made it, but now there is a new error  >:(




13>------ Build started: Project: libsvn_ra_neon, Configuration: Release Win32 ------
13>Compiling...
13>commit.c
12>zlibstat.lib(inflate.obj) : error LNK2019: unresolved external symbol _inflate_fast referenced in function _inflate
12>..\..\..\Release\subversion\libsvn_subr\libsvn_subr-1.dll : fatal error LNK1120: 1 unresolved externals
12>Build log was saved at "file://c:\build\subversion-1.6.17-x86\Release\subversion\libsvn_subr\BuildLog.htm"
12>libsvn_subr_dll - 2 error(s), 2 warning(s)
14>------ Build started: Project: libsvn_delta_dll, Configuration: Release Win32 ------
14>Performing Custom Build Step
14>Compiling...
14>empty.c
13>util.c
14>Compiling resources...
14>Microsoft (R) Windows (R) Resource Compiler Version 6.1.6723.1
14>Copyright (C) Microsoft Corporation.  All rights reserved.
14>Linking...
14>   Creating library ..\..\..\Release\subversion\libsvn_delta\libsvn_delta-1.lib and object ..\..\..\Release\subversion\libsvn_delta\libsvn_delta-1.exp
13>session.c
14>zlibstat.lib(inflate.obj) : error LNK2019: unresolved external symbol _inflate_fast referenced in function _inflate
14>..\..\..\Release\subversion\libsvn_delta\libsvn_delta-1.dll : fatal error LNK1120: 1 unresolved externals
14>Build log was saved at "file://c:\build\subversion-1.6.17-x86\Release\subversion\libsvn_delta\BuildLog.htm"
14>libsvn_delta_dll - 2 error(s), 0 warning(s)
13>replay.c
13>props.c
21>------ Build started: Project: libsvn_ra_dll, Configuration: Release Win32 ------
20>wc_db.c
21>Performing Custom Build Step
21>Compiling...
21>empty.c
21>Compiling resources...
21>Microsoft (R) Windows (R) Resource Compiler Version 6.1.6723.1
21>Copyright (C) Microsoft Corporation.  All rights reserved.
21>Linking...
20>util.c
21>   Creating library ..\..\..\Release\subversion\libsvn_ra\libsvn_ra-1.lib and object ..\..\..\Release\subversion\libsvn_ra\libsvn_ra-1.exp
21>libneon.lib(inflate.obj) : error LNK2019: unresolved external symbol _inflate_fast referenced in function _inflate
21>..\..\..\Release\subversion\libsvn_ra\libsvn_ra-1.dll : fatal error LNK1120: 1 unresolved externals
21>Build log was saved at "file://c:\build\subversion-1.6.17-x86\Release\subversion\libsvn_ra\BuildLog.htm"
21>libsvn_ra_dll - 2 error(s), 0 warning(s)
20>update_editor.c

mario

Well now I'm confused! the x64 build is ok! Gonna upload it tomorrow and do the release. I wonder why the x86 build still fails. Too tired to figure out anything now  ;)

Sob

#56
It's not a new error, it's our old troublemaker _inflate_fast from zlib. Maybe you forgot to change build_zlib.bat for x86 build? I got the same errors without it, but it went away with the modification.

mario

Done! Downloads will be very soon on the download page!

mario

The new 1.7 RC 3 can be found on the download page. It is recomment NOT to use in this in productive systems, only for testing

mario

Can someone please test [LINK removed]
I get the error 0xc000007b when running. Even when the build is without any issues. So I think it is an issue with the build system to run it.

with 1.6.15r2 kmradke told it works fine, but that also fails for me with the same issue.
Quote from: kmradke on December 21, 2010, 05:51:37 AM
64-bit version appears to work great!

Thanks!

Thanks a lot for testing