Gallio runs MbUnit tests very slowly?
Currently I’m looking close to one interesting project called Subtext. It being developed and managed by Phil Haack and recently he announced release of version 2.5. It open-souce so anyone is able to get SVN snapshot and digg into code and
even to contribute to this project. I would like to do a separate blog post on Subtext, but now I want to touch another topic.
Subtext project uses MbUnit test framework, that I have no chance to work before. So as my first goal I started to look for tools to work with MbUnit and ways of its integration to Visual Studio. Obvious choice was a Gallio. Gallio is open-source automation test framework, a bundle contains *maybe* all known unit-test systems (NUnit, MbUnit, NCover, TypeMock etc.) as well as add-on to Visual Studio that allows to run MbUnit tests by Testdriven.net runner.
I was preatty happy to find that one exists. I’ve installed the version that official page refered to Gallio v3.1 Update 2 build 397 - x86. Installed on VS 2008 no problem and I’ve started to play with Subtext tests. But I’ve noticed that it works very slow, namely run of particular test takes about 20 sec on my machine. That is too much time for ordinar test. At first I blamed Subtext that it performs test database initialization in [SetUp] function of test assembly. But I just commented out this initialization I saw that situation has not changed.
I googled a bit I found that Oliver Sturm also experienced same problem and did a post on his blog. He did a great job and noticed that reason for such slow down was gallio.utility.exe application that hangs for about 20 sec on each start. Tests are using that for initialization so it slows down each start of tests. He had created a bug report and as Jeff Brown the problem was fixed for v3.2.380. I went to a distribution site and downloaded latest 3.2 vesrion. I’ve installed it and try to run test. Woow, it was run for
just a second! Good news I thought! But when I tried to run all tests from a test assembly I was disappointed that it simply does not work. It says about exception during initialization of Gallio. I reported on that with new issue to Gallio.
I’ve decied to try earlier verions of 3.2.. But still had the same problem. So, 3.2 realease does not work on my machine at all and do not know the reason. My only choice was to back to 3.1 but try some more recent daily builds. I’ve got Version 3.1 build 514 and finally it started to work as expected. Test case run took about 4 seconds, slower that 3.2 456 but anyway, not so bad as for 3.1 397.
Gallio is preatty good software, so I really recommed to to check it out. 3.1 514 works fine on Visual Studio 2008.
Hope this one will be useful to one who is going to start with Gallio or met same problem as me.
Currently I’m looking close to one interesting project called Subtext. It being developed and managed by Phil Haack and recently he announced release of version 2.5. It open-souce so anyone is able to get SVN snapshot and digg into code and
even to contribute to this project. I would like to do a separate blog post on Subtext, but now I want to touch another topic.
Subtext project uses MbUnit test framework, that I have no chance to work before. So as my first goal I started to look for tools to work with MbUnit and ways of its integration to Visual Studio. Obvious choice was a Gallio. Gallio is open-source automation test framework, a bundle contains *maybe* all known unit-test systems (NUnit, MbUnit, NCover, TypeMock etc.) as well as add-on to Visual Studio that allows to run MbUnit tests by Testdriven.net runner.
I was preatty happy to find that one exists. I’ve installed the version that official page refered to Gallio v3.1 Update 2 build 397 - x86. Installed on VS 2008 no problem and I’ve started to play with Subtext tests. But I’ve noticed that it works very slow, namely run of particular test takes about 20 sec on my machine. That is too much time for ordinar test. At first I blamed Subtext that it performs test database initialization in [SetUp] function of test assembly. But I just commented out this initialization I saw that situation has not changed.
I googled a bit I found that Oliver Sturm also experienced same problem and did a post on his blog. He did a great job and noticed that reason for such slow down was gallio.utility.exe application that hangs for about 20 sec on each start. Tests are using that for initialization so it slows down each start of tests. He had created a bug report and as Jeff Brown the problem was fixed for v3.2.380. I went to a distribution site and downloaded latest 3.2 vesrion. I’ve installed it and try to run test. Woow, it was run for just a second! Good news I thought! But when I tried to run all tests from a test assembly I was disappointed that it simply does not work. It says about exception during initialization of Gallio. I reported on that with new issue to Gallio.
I’ve decied to try earlier verions of 3.2.. But still had the same problem. So, 3.2 realease does not work on my machine at all and do not know the reason. My only choice was to back to 3.1 but try some more recent daily builds. I’ve got Version 3.1 build 514 and finally it started to work as expected. Test case run took about 4 seconds, slower that 3.2 456 but anyway, not so bad as for 3.1 397.
Gallio is preatty good software, so I really recommed to to check it out. 3.1 514 works fine on Visual Studio 2008.
Hope this one will be useful to one who is going to start with Gallio or met same problem as me.
Subtext project uses MbUnit test framework, that I have no chance to work before. So as my first goal I started to look for tools to work with MbUnit and ways of its integration to Visual Studio. Obvious choice was a Gallio. Gallio is open-source automation test framework, a bundle contains *maybe* all known unit-test systems (NUnit, MbUnit, NCover, TypeMock etc.) as well as add-on to Visual Studio that allows to run MbUnit tests by Testdriven.net runner.
I was preatty happy to find that one exists. I’ve installed the version that official page refered to Gallio v3.1 Update 2 build 397 - x86. Installed on VS 2008 no problem and I’ve started to play with Subtext tests. But I’ve noticed that it works very slow, namely run of particular test takes about 20 sec on my machine. That is too much time for ordinar test. At first I blamed Subtext that it performs test database initialization in [SetUp] function of test assembly. But I just commented out this initialization I saw that situation has not changed.
I googled a bit I found that Oliver Sturm also experienced same problem and did a post on his blog. He did a great job and noticed that reason for such slow down was gallio.utility.exe application that hangs for about 20 sec on each start. Tests are using that for initialization so it slows down each start of tests. He had created a bug report and as Jeff Brown the problem was fixed for v3.2.380. I went to a distribution site and downloaded latest 3.2 vesrion. I’ve installed it and try to run test. Woow, it was run for just a second! Good news I thought! But when I tried to run all tests from a test assembly I was disappointed that it simply does not work. It says about exception during initialization of Gallio. I reported on that with new issue to Gallio.
I’ve decied to try earlier verions of 3.2.. But still had the same problem. So, 3.2 realease does not work on my machine at all and do not know the reason. My only choice was to back to 3.1 but try some more recent daily builds. I’ve got Version 3.1 build 514 and finally it started to work as expected. Test case run took about 4 seconds, slower that 3.2 456 but anyway, not so bad as for 3.1 397.
Gallio is preatty good software, so I really recommed to to check it out. 3.1 514 works fine on Visual Studio 2008.
Hope this one will be useful to one who is going to start with Gallio or met same problem as me.