Wednesday, January 6, 2010

Building MySQL Cluster 7.0 on Windows

This walkthrough will show you how to configure your more or less clean install of Windows(you can use XP, Vista or 7) in order to build MySQL Cluster on Windows. The setup also works for the vanilla MySQL Server since the only real difference is the configure switches used to activate build of the NDB storage engine and the NDBCLUSTER handler.

Installing the necessary tools
  • Install bison from GnuWin32 - http://gnuwin32.sourceforge.net/packages/bison.htm, this program is used generate the yacc parser in MySQL Server and is invoked automatically by a build rule. To avoid problem's with spaces in the path name, I recommend installing it to non default location c:\GnuWin32(i.e remove the "Program Files (x86)" part) - this is luckily the only tools that seem to have problem with spaces in the path.
  • Install diff from GnuWin32 - http://gnuwin32.sourceforge.net/packages/diffutils.htm, this program is used by the test programs (to show the difference between the actual and expected result if a .test file run by mysql-test-run.pl should fail) and also to generate a somewhat more advanced diff for commit mails. This package will also go into c:\GnuWin32(the installer will autodetect it).
  • Add the bin directory of your GnuWin32 installation to your PATH to make both of the above programs executable from command line. (i.e append ";c:\GnuWin32\bin")
  • Install Visual Studio Express 2008 - http://www.microsoft.com/Express/VC/, normally you can skip installing any extras like Silverlight and Microsoft SQL Server.
  • Install Cmake from http://www.cmake.org/, there is a brand new version 2.8 and it seems to work fine. Tell installer to add cmake to your PATH.
  • Install the version control tool Bazaar -http://wiki.bazaar.canonical.com/WindowsDownloads - using the standalone package installer. Use default settings to make it add itself to PATH.

Getting the source using bzr
NOTE! You can also download a "source release" aka. "source dist" from http://dev.mysql.com/downloads/cluster/ and select "Source Code" as your platform and then download the "Genric Linux (Architecture independent), Compressed TAR Archive", unpack and build. But since this is the hardcore instructions I'm using Bazaar to do some real development.

The public branch of MySQL Cluster 7.0 is hosted on launchpad, it's a readonly mirror of our internal repository and the details about the branch can be found at https://code.launchpad.net/~mysql/mysql-server/mysql-cluster-7.0 . For example it's possible to see the latest pushes and check the diff and commit messages for each push.

  • Open a command prompt and change to the directory where you like to keep your MySQL clones. Normally you use one clone for each task you work on so it's good to initialize a shared repository. It's very important to use the option --pack-0.92 to create a repository using the same format as the MySQL repositories, otherwise bzr will try to convert from our "old" format to the latest and that will use up all your RAM and CPU for quite some time(haven't heard anyone that been patient enough to wait for it to complete).

    M:\> bzr init-repo --pack-0.92 .
  • The branch location is lp:~mysql/mysql-server/mysql-cluster-7.0 and it may unfortunately take some time to do the first branch over the internet so be patient. Unless you are logged in to launchpad, you will get a complaint from bzr - just ignore it(although rumors say that something called "streaming" will be turned on if you do login and that should speed up the cloning).

    M:\> bzr branch lp:~mysql/mysql-server/mysql-cluster-7.0 7.0
  • Change into the newly branched tree and run the "windows configure", a small javascript that will generate a file that tells cmake which parts to include into the build. I'm showing tree different configure lines below and you should only select one of them. The first one is just MySQL Server with NDB, the second one adds all our test programs for MySQL Cluster and finally the third is the configure line we're curently using to build the test builds in our continous integrion system.

    M:\> cd 7.0
    M:\7.0> cscript win\configure.js WITH_NDBCLUSTER_STORAGE_ENGINE
    M:\7.0> cscript win\configure.js WITH_NDBCLUSTER_STORAGE_ENGINE WITH_NDB_TEST
    M:\7.0> cscript win\configure.js WITH_INNOBASE_STORAGE_ENGINE WITH_PARTITION_STORAGE_ENGINE WITH_ARCHIVE_STORAGE_ENGINE WITH_BLACKHOLE_STORAGE_ENGINE WITH_EXAMPLE_STORAGE_ENGINE WITH_FEDERATED_STORAGE_ENGINE WITH_EMBEDDED_SERVER WITH_NDBCLUSTER_STORAGE_ENGINE WITH_NDB_TEST __NT__ "COMPILATION_COMMENT=MySQL Cluster test build XXX"

  • Now it's time to make Cmake do it's magic and generate the Visual Studio project files. It will create a solution file called MySQL.sln and one project for every binary and lib we have selected to build.
  • M:\7.0> cmake -G "Visual Studio 9 2008"
  • Open the resulting MySQL.sln in Visual Studio. By default the "Active Configuration" will be "Debug" and although it now works to run the binaries generated with debug, my recomendation is still to switch to "RelWithDebinfo". Hopefully everything works and the build succeeds. The most common problem here is that it does not find bison and you get a build failure for sql_yacc.cc, in that case just check bison is in your path.