Functional testing by javascript with FuncUnit
Functional tests are something that is better to use as early as possible, to get any valuable results. Obvious choice for ASP.net applications are Seleminum for .NET, WatiN or SpecFlow. But all of them are supposed to use C# as programming language and run their tests by special runners. Thought the experience I saw how cool is to write tests with full dynamic languages, like javascript and be able to run tests directly from browser, so I can debug some broken functionality. As I saw FuncUnit project and it really attracted my attention! Here I describe some initial experince of testing ASP.net MVC application with FuncUnit.
What it is all about?
FuncUnit is very compact and elegant functional testing framework. It combines the power of jQuery, qUnit, Selemium, Synthetic. Later it was a part of javascriptMVC framework, but now it is available as separate framework. Major features are:
- It is only javascript
- Could run tests in browser mode and command line by means of Selemium
- Simple API
- Easy to debug
- Automate your existing qUnit tests
- Run file mode and server page mode
Yes, it simple.. just from the reading of documentation and downloading the package you are ready to start you testing. I’ve managed to cover mostly all functionality of my small application in 3 hours or so.
Integration
Download FuncUnit package here.
It contains all required javascript files, as well as selemium jar's, so as you
have Java runtime installed on your machine you immediately able to run tests
from command line. FuncUnit does not say what particular framework should be
behind of your application, it would work with any.
It requires that tests page are on the same level as tested page. At the beginning it was confusing me, because in ASP.net MVC we don’t have pages at all, only controllers and corresponding actions; my existing qUnit tests were placed to Scripts/Tests/index.html runner and I expected to have same layout. But it have to be changed a little for FuncUnit. So, filemode testing is not really applicable for ASP.net MVC, you should go with server pages mode.
I placed both FuncUnit and qUnit test pages at the root of application. Test code itself is referenced from Scripts/Tests/acceptance and Scripts/Tests/acceptance.
Content of FuncUnit placed to Scripts/Test/framework, so I got something like:
Test page
It is very similar (even identical) to qUnit page. Just reference framework and tests scripts.
<html>
<head>
<title>Trackyt.net Functional Test</title>
<link rel="stylesheet" type="text/css" href="http://v3.javascriptmvc.com/funcunit/dist/qunit.css" />
<script type='text/javascript' src='Scripts/Tests/framework/funcunit/funcunit.js'></script>
<!-- Tests -->
<script type='text/javascript' src='Scripts/Tests/acceptance/tests.home.js'></script>
<script type='text/javascript' src='Scripts/Tests/acceptance/tests.signup.js'></script>
<script type='text/javascript' src='Scripts/Tests/acceptance/tests.bugs.js'></script>
</head>
<body>
<h1 id="qunit-header">FuncUnit Test Suite</h1>
<h2 id="qunit-banner"></h2>
<div id="qunit-testrunner-toolbar"></div>
<h2 id="qunit-userAgent"></h2>
<ol id="qunit-tests"></ol>
</body>
</html>
* This source code was highlighted with Source Code Highlighter.
Test code
As I said its fun and easy to write tests with FuncUnit. You have everything that should be in functional testing framework: open pages, read values and put values to controls, wait for events.. clicking and dragging mouse. Tests code/layout is the same as you get used with qUnit. Each test basically does:
-
Open test page, by
S.open("page_url_from_root") (ex. S.open(“Admin/Login”), will run http://localhost/app/admin/login)
-
Wait for some html elements appear in browser, with
S.wait();
-
Type some values to control
S('#myControlId').type('bla-bla') and read values S('#anotherControlId').val();
-
Assert for results with
ok, same, equal methods;
I’ll do several example from my code.
Simple one, I check that user is not able to login with empty password:
// I as user could not login with empty password
test("password is empty", function () {
// arrange
S('#Email').type('a@a.com');
// act
S('#submit-button').click();
// assert
S('#PasswordValidationMessage').visible(function () {
var message = S('#PasswordValidationMessage li:nth-child(1)').html();
ok(message == "Password is empty", "I as user could not login with empty password");
});
});
* This source code was highlighted with Source Code Highlighter.
Or that I’m not able to register with already registered email:
// I as user could not register with same email twice
test("register twice", function () {
// arrange
var email = "test" + new Date().getSeconds() + new Date().getMilliseconds() + "@trackyt.net";
S('#Email').type(email);
S('#Password').type(email);
S('#ConfirmPassword').type(email);
S('#submit-button').click(function () {
// wait till registration is done
S.wait();
// act
// go back to register page and try to register with same credentials
S.open("Registration", function () {
S('#Email').type(email);
S('#Password').type(email);
S('#ConfirmPassword').type(email);
S('#submit-button').click(function () {
// assert
S('#PasswordValidationMessage').visible(function () {
var message = S('.validation-summary-errors li:nth-child(1)').html();
same(message, "Sorry, user with such email already exist. Please register with different email.", "I as user could not register with same email twice");
});
});
});
});
});
* This source code was highlighted with Source Code Highlighter.
To more complex, bug reproducing issues:
// https://github.com/alexbeletsky/Trackyourtasks.net/issues/#issue/20
test("tasks are disappeared", function () {
// go to sign up page
S.open("Registration");
// create new account
var email = "test_bugs" + new Date().getSeconds() + new Date().getMilliseconds() + "@trackyt.net";
S('#Email').type(email);
S('#Password').type(email);
S('#ConfirmPassword').type(email);
S('#submit-button').click(function () {
S('#tasks').exists(function () {
// create several tasks
S('#task-description').click().type("fix issue 20");
S('#add-task').click();
S.wait(500);
S('#task-description').click().type("fix issue 20 2");
S('#add-task').click();
S.wait(500);
// and log off of dashboard
S('#sign-out').click();
});
});
// now sign in with same account
S.open("Login");
S('#Email').type(email);
S('#Password').type(email);
S('#submit-button').click(function () {
// wait till tasks are ready
S('#tasks').exists(function () {
S.wait(function () {
var tasks = S('.task').size();
// tasks created before must exist
ok(tasks == 2, "tasks created before must exist");
});
});
});
});
* This source code was highlighted with Source Code Highlighter.
I found that those tests are also a kind of documentation! I could document user stories with such tests and maybe even use tools to generate documentation just from tests code.
Running tests from browser
While development it is more preferable to run tests directly from browser. To do that I just select FunctionalTests.html file in Solution Explorer and press Ctrl+Shift+W that mean “View in browser”. Please note, that FuncUnit tests are run in pop-up window, so you have to enable pop ups in your browser. Browser will open http://localhost/virtualdir/FunctionalTests.html and tests will run. For more convenience I also put a bookmark for test page in browser, so I could always re-run all tests by one click.
Running tests from command-line
Major feature of FuncUnit is of cause it’s ability to run from command-line, so you can easily integrate to your continuous integration system. There a script envjs.bat that would easily to start the tests under Selenium and provide with results as command line output (you can also run qUnit tests from command line with envjs.bat).
From the root of web application, I start:
.\Scripts\Tests\framework\funcunit\envjs.bat http://localhost/tracky/FunctionalTests.html
BTW, there is a small issue in envjs.bat that stops me to successfully run it initially, it does not take into accounts that it could be run as envjs.bat, but simpy as envjs. Please see my version of this file here.
Deployment
Sure you don’t want to make those tests public on your site. There are 2 options: change Web.config to disable users to access FunctionalTets.html, UnitTests.html or just delete those 2 before deployment to production. Since I do simple xcopy deployment, option 2 works for me.
Demonstration
To make picture full, I prepared small screencast that would demonstrate how I used the framework. I like it, it fits my requirements (at least for now). I hope you also like it and find this information useful! And also, you should check the repository for test code examples.
Functional tests are something that is better to use as early as possible, to get any valuable results. Obvious choice for ASP.net applications are Seleminum for .NET, WatiN or SpecFlow. But all of them are supposed to use C# as programming language and run their tests by special runners. Thought the experience I saw how cool is to write tests with full dynamic languages, like javascript and be able to run tests directly from browser, so I can debug some broken functionality. As I saw FuncUnit project and it really attracted my attention! Here I describe some initial experince of testing ASP.net MVC application with FuncUnit.
What it is all about?
FuncUnit is very compact and elegant functional testing framework. It combines the power of jQuery, qUnit, Selemium, Synthetic. Later it was a part of javascriptMVC framework, but now it is available as separate framework. Major features are:
- It is only javascript
- Could run tests in browser mode and command line by means of Selemium
- Simple API
- Easy to debug
- Automate your existing qUnit tests
- Run file mode and server page mode
Yes, it simple.. just from the reading of documentation and downloading the package you are ready to start you testing. I’ve managed to cover mostly all functionality of my small application in 3 hours or so.
Integration
Download FuncUnit package here. It contains all required javascript files, as well as selemium jar's, so as you have Java runtime installed on your machine you immediately able to run tests from command line. FuncUnit does not say what particular framework should be behind of your application, it would work with any.
It requires that tests page are on the same level as tested page. At the beginning it was confusing me, because in ASP.net MVC we don’t have pages at all, only controllers and corresponding actions; my existing qUnit tests were placed to Scripts/Tests/index.html runner and I expected to have same layout. But it have to be changed a little for FuncUnit. So, filemode testing is not really applicable for ASP.net MVC, you should go with server pages mode.
I placed both FuncUnit and qUnit test pages at the root of application. Test code itself is referenced from Scripts/Tests/acceptance and Scripts/Tests/acceptance.
Content of FuncUnit placed to Scripts/Test/framework, so I got something like:
Test page
It is very similar (even identical) to qUnit page. Just reference framework and tests scripts.
<html>
<head>
<title>Trackyt.net Functional Test</title>
<link rel="stylesheet" type="text/css" href="http://v3.javascriptmvc.com/funcunit/dist/qunit.css" />
<script type='text/javascript' src='Scripts/Tests/framework/funcunit/funcunit.js'></script>
<!-- Tests -->
<script type='text/javascript' src='Scripts/Tests/acceptance/tests.home.js'></script>
<script type='text/javascript' src='Scripts/Tests/acceptance/tests.signup.js'></script>
<script type='text/javascript' src='Scripts/Tests/acceptance/tests.bugs.js'></script>
</head>
<body>
<h1 id="qunit-header">FuncUnit Test Suite</h1>
<h2 id="qunit-banner"></h2>
<div id="qunit-testrunner-toolbar"></div>
<h2 id="qunit-userAgent"></h2>
<ol id="qunit-tests"></ol>
</body>
</html>
* This source code was highlighted with Source Code Highlighter.Test code
As I said its fun and easy to write tests with FuncUnit. You have everything that should be in functional testing framework: open pages, read values and put values to controls, wait for events.. clicking and dragging mouse. Tests code/layout is the same as you get used with qUnit. Each test basically does:
-
Open test page, by
S.open("page_url_from_root")(ex. S.open(“Admin/Login”), will run http://localhost/app/admin/login) -
Wait for some html elements appear in browser, with
S.wait(); -
Type some values to control
S('#myControlId').type('bla-bla')and read valuesS('#anotherControlId').val(); -
Assert for results with
ok, same, equalmethods;
I’ll do several example from my code.
Simple one, I check that user is not able to login with empty password:
// I as user could not login with empty password
test("password is empty", function () {
// arrange
S('#Email').type('a@a.com');
// act
S('#submit-button').click();
// assert
S('#PasswordValidationMessage').visible(function () {
var message = S('#PasswordValidationMessage li:nth-child(1)').html();
ok(message == "Password is empty", "I as user could not login with empty password");
});
});
* This source code was highlighted with Source Code Highlighter.Or that I’m not able to register with already registered email:
// I as user could not register with same email twice
test("register twice", function () {
// arrange
var email = "test" + new Date().getSeconds() + new Date().getMilliseconds() + "@trackyt.net";
S('#Email').type(email);
S('#Password').type(email);
S('#ConfirmPassword').type(email);
S('#submit-button').click(function () {
// wait till registration is done
S.wait();
// act
// go back to register page and try to register with same credentials
S.open("Registration", function () {
S('#Email').type(email);
S('#Password').type(email);
S('#ConfirmPassword').type(email);
S('#submit-button').click(function () {
// assert
S('#PasswordValidationMessage').visible(function () {
var message = S('.validation-summary-errors li:nth-child(1)').html();
same(message, "Sorry, user with such email already exist. Please register with different email.", "I as user could not register with same email twice");
});
});
});
});
});
* This source code was highlighted with Source Code Highlighter.To more complex, bug reproducing issues:
// https://github.com/alexbeletsky/Trackyourtasks.net/issues/#issue/20
test("tasks are disappeared", function () {
// go to sign up page
S.open("Registration");
// create new account
var email = "test_bugs" + new Date().getSeconds() + new Date().getMilliseconds() + "@trackyt.net";
S('#Email').type(email);
S('#Password').type(email);
S('#ConfirmPassword').type(email);
S('#submit-button').click(function () {
S('#tasks').exists(function () {
// create several tasks
S('#task-description').click().type("fix issue 20");
S('#add-task').click();
S.wait(500);
S('#task-description').click().type("fix issue 20 2");
S('#add-task').click();
S.wait(500);
// and log off of dashboard
S('#sign-out').click();
});
});
// now sign in with same account
S.open("Login");
S('#Email').type(email);
S('#Password').type(email);
S('#submit-button').click(function () {
// wait till tasks are ready
S('#tasks').exists(function () {
S.wait(function () {
var tasks = S('.task').size();
// tasks created before must exist
ok(tasks == 2, "tasks created before must exist");
});
});
});
});
* This source code was highlighted with Source Code Highlighter.I found that those tests are also a kind of documentation! I could document user stories with such tests and maybe even use tools to generate documentation just from tests code.
Running tests from browser
While development it is more preferable to run tests directly from browser. To do that I just select FunctionalTests.html file in Solution Explorer and press Ctrl+Shift+W that mean “View in browser”. Please note, that FuncUnit tests are run in pop-up window, so you have to enable pop ups in your browser. Browser will open http://localhost/virtualdir/FunctionalTests.html and tests will run. For more convenience I also put a bookmark for test page in browser, so I could always re-run all tests by one click.
Running tests from command-line
Major feature of FuncUnit is of cause it’s ability to run from command-line, so you can easily integrate to your continuous integration system. There a script envjs.bat that would easily to start the tests under Selenium and provide with results as command line output (you can also run qUnit tests from command line with envjs.bat).
From the root of web application, I start:
.\Scripts\Tests\framework\funcunit\envjs.bat http://localhost/tracky/FunctionalTests.html
BTW, there is a small issue in envjs.bat that stops me to successfully run it initially, it does not take into accounts that it could be run as envjs.bat, but simpy as envjs. Please see my version of this file here.
Deployment
Sure you don’t want to make those tests public on your site. There are 2 options: change Web.config to disable users to access FunctionalTets.html, UnitTests.html or just delete those 2 before deployment to production. Since I do simple xcopy deployment, option 2 works for me.
Demonstration
To make picture full, I prepared small screencast that would demonstrate how I used the framework. I like it, it fits my requirements (at least for now). I hope you also like it and find this information useful! And also, you should check the repository for test code examples.