Alexander Beletsky's development blog

My profession is engineering

How to start using Git in SVN-based organization

I’ve been using Git about a year now, immediately after creation my github account. I got really nice impression of using this tool, but still - on my primary job at e-conomic we are using SVN as primary VCS tool.

You are probably aware about all good thing with Git. If you still don’t you might check out these great answers on Stackoverflow. In short - Git adds benefits, but it adds complexity as well.

After a bit evaluation I found that those benefits are worth to stop using SVN and switch to Git. But, if you ask your colleagues “Are you ready to go with Git?” you would probably hear “No!”. It is clear why, people are getting used to current tools.. infrastructure are build around them. And switching from one VCS to another seemed as destruction of building fundament for many developers.

Fortunately, Git has a killer feature that other DVCS does not have! And this feature will help you to smoothly start with Git even if your company is still using SVN as standard. Yes, I’m talking about git-svn.

Believe me, it is amazing! I always have a little distrust for such kind of integration tool, but so far it works as charm. So, I’ll give you a small instructions that will make you possible to get to your office tomorrow and start use Git as your own VCS.

Setup you environment

  • Download and install msysgit
  • Setup local user information, example
        git config --global user.name “alexander.beletsky”
        git config --global user.email “alexander.beletsky@gmail.com”
    
  • If you are on Windows and doing only-Windows development, it is recommended to turn off autocrlf
        git config core.autocrlf false
    

Checking out code from SVN repository

  • Run git clone command (it might take some time, depending on repository and history size)
        git svn clone https://mysvnserver/repo
    

As clone finished, you will see that new folder repo created with the content of your SVN repository. Now, this folder is actually Git repository + you local sources copy. Try to run git log inside the repository and you will see that all your history are there!

Let’s do some work

Now, suppose you start to work on some bugfix. By default, git repository contains one branch called master. Good style is do not do direct commits into master at all, just keep it as synchronization point between SVN repository and your local repository. So, to do actual job you to create branch.

    git checkout -b bugfix-id-1453

This command will create new branch called bugfix-id-1453 and immediately switch to it. If you haven’t noticed how that happened - please welcome to git’s high performance world.

You changed some file and ready to commit. In git it’s a 2 steps operation. First, you need to put content to stage.. second, you store the content in new commit object.

    git add .
    git commit -m "issue has been fixed"

Commit changes back to SVN

You’ve fixed a bug and now fixed is placed in bugfix-id-1453. But you should give it back to rest of the world.

Checkout your master branch:

    git checkout master

Get the latest changes from SVN:

    git svn rebase

Now SVN server and local master branch are identical. We have to merge our fix into master:

    git merge bugfix-id-1453

Now master contains previous SVN state + new fix. We have to synchronize local repository and SVN server:

    git svn dcommit

Conclusions

Git-svn is perfect tool to start using git without radical changes in organization. It gives everybody change to try it out, how much it really fits your needs. I’m having a big pleasure now of working with Git. I’m not saying it’s ideal smooth process, sure I met some difficulties. But for me those difficulties are nothing comparing the benefits I’m getting with it.

Thinking about cloud..

cloud

I’m really inspired with cloud ideas since I first time met it. No doubt, that clouds are future of computing. I like the metaphor comparing cloud and electric power plant: “Last century if you plan your business depend on electricity, you have to afford huge and expensive electric generator - you have to host it, support it and do all maintenance stuff by yourself. You pay for everything by yourself, just to be able to plug one device. Then power plant come and electric wires covered a country, you no longer need your own electric generator, you can buy the electricity from power plant station. You pay only for that you used and it is much cheeper, since a lot of people using it together with you”. Clouds are modern power plants - to be online you no longer need to buy servers, establish a data center, maintenance all infrastructure. You could by the resources from cloud services provider.

Clouds are about to change the business. It is much more easy to start-up new product in web, since you have to care only about product, not infrastructure. I knew some companies in past with quite good ideas, but those ideas buried under infrastructure complexity and expensiveness. And the clouds are about to solve that problem, microISV should be happy about.

I was thinking about all my current and future products are going to in cloud for sure. First and obvious point to look was Microsoft Azure. But after some elaboration I disappointed a bit:

  1. Azure is overcomplicated. I’m not saying I’m not able to understand that.. But all those Roles, Processes, Blobs - it is just too much details. But what I basically waht is to place my existing working product into elastic space, that’s it.
  2. It is still expensive. This Azure price calculator, my simplest configuration would be ~50 USD/month. This is more than I currently pay for VPS.

So, I saw not enough reasons to switch immediately.

Yesterday, I’ve been contacted by Host1Plus company. They started out beta cloud in the new datacenter which is located in Frankfurt Main, Germany. And they are looking for beta testers of new service. The proposition looks really fine to me, since I could try the power of clouds and in the same time help a company with a feedback I could provide.

I’m going to join. If it is interesting for you, join with me. Here is some links that could be helpful:

Inside ASP.NET MVC: All begins here - MvcHanlder

Last time we explored a way of HTTP request from the IIS up to MvcHanlder. Then UrlRouting module match URL we registered in Application_Start(), it gets corresponding IRouteHandler class and calls for GetHttpHandler method. In ASP.NET MVC URL IRouteHandler interface is implemented by MvcRouteHandler class which return MvcHanlder instance. The actual MVC story begins exactly in MvcHanlder internals.

Generic overview

We know that HttpHandlers are special objects that handles requests and produce a response. MvcHanlder produce nothing by itself. Instead, it uses ControllerBuilder object for creating Controller and call controller Execute method. It is Execute method responsibility to find corresponding action, call action and wrap action return result it HTTP response.

mvc handler declaration

As you can see, it implements 3 interfaces: IHttpHandler, IHttpAsyncHandler and IRequiresSessionState. The handler could operate in 2 modes: sync and async, depending on caller. ASP.NET frameworks ProcessRequestInternal method try to utilize asynchronous mode of work first.

The constructor of MvcHanlder receives RequestContext class, that is part of System.Web.Routing and contains information about HttpContext.

Synchronous and Asynchronous modes

Of course, the details of operation in synchronous and asynchronous modes are differ from each other, but main algorithm of work is exactly the same.. Let’s first see how it operates in sync mode.

Process the request

Due to IHttpHandler interface, MvcHanlder expected to support one property and one method:

mvc handler declaration

IsReusable is always false, meaning that for each request the runtime would create new instance of handler. ProcessRequest receives HttpContext and basically delegates call to internal ProcessRequest method.

mvc handler declaration

Notice to the first line.. HttpContext is being wrapped with HttpContextWrapper and casted to HttpContexBase. Rest of code never (mostly) using HttpContext directly, but through HttpContexBase. The HttpContexBase have been introduced to improve testability of ASP.NET applications, since HttpContext is unable to be mocked.

ControllerBuilder, ControllerFactory, Controller

Let’s see inside the ProcessRequest method to see how exactly the controller is being constructed and called.

mvc handler declaration

The call is wrapped with a little security handler, that would do perform the check for Trust settings, before calling internal action. Inside the action IController and IControllerFactory are being declared and initialized in ProcessRequestInit method.

mvc handler declaration

First, it puts MVC version information into HTTP response header, removes optional parameters from routes (one we mark with UrlParameter.Optional) and gets the name of controller from RequestContext. Remember, in global.asax we define URL pattern like “{controller}/{action}/{index}”, so in case of such URL products/search/122 the GetRequiredString will return “products”.

ControllerBuilder is a singleton and factory method that creates IControllerFactory instance.

ControllerFactory is responsible for creation of controller by given name.

The ProcessRequestInit gets the factory from ControllerBuilder, ask factory to create controller by the name from URL and returns results back to ProcessRequest.

Here we also could see good example of comments that lies.. “// Instantiate the controller and call Execute” - maybe sometime ago that was true, but not its false since Execute is not called here.

Execution

As soon as controller factory and controller are successfully created, ProcessRequest calls for controller Execute method. Now, it is controller responsibility to produce the output and we will see how it does that in next “Inside MVC” posts. Execution is wrapped inside try / finally block and finalize code will release the controller by calling ReleaseContoller method of factory.

Meanwhile in Asynchronous…

In async world the things are bit more complex. There are no one single method, but 2 instead. BeginProcessRequest and EndProcessRequest.

The request initialization and creation of factory and controller is exactly the same. The difference in execution, for async controller 2 delegates are being created,

mvc handler declaration

In fact, that than the request execution begins, beginDelegate is called that would call BeginExecute of the controller. At the end of request endDelegate called that would call EndExecute of controller.

What about tests?

That was a great surprise to me that MvcHandler is well unit tested. All aspects of its work are covered with corresponding tests.. I’ve heard a lot of developers complains like, “oh, it is so low-level code.. could not test it”. But MVC team, proof it is wrong. HttpContexBase could be easy mocked, so no excuse of not writing unit tests for modules/handlers. As I said in into, test code is really clean, following AAA principle’s, so I got a lot of pleasure of looking inside.

BTW, they are using Moq framework for mocking, that I personally like as well :).

Previous: Inside ASP.NET MVC: Route to MvcHandler Next: Inside ASP.NET MVC: ControllerBuilder class

Candidate, Application I Made on Hackatone

Here is quick description of application application I made on Hackatone this weekend.

Candidate.net is just working name for the project that I’m going to change as soon as I got better idea :). It supposed to be a kind of service that provides integration with your github repositories and provides a Continues Deployment cycle for that repositories. Namely, every code change that being pushed to github is immediately build, tested and deployed, so the time between code push and production is very short excludes any manual work.

I’ve spent ~16 hours for building up this application, sure I haven’t managed to make it as a service.. but - during the process I’ve changed my vision for project, so first I would like to create good continues build engine (like Team City or Jenkins) and base my future service on top of it (yeah, I already got some further plans :).

Technologies behind:

  • ASP.NET MVC3
  • jQuery + some small pluggins
  • UppercuT build framework

A picture says 1000 words, so here my little screencast of Candidate.net in action. It demonstrates the ability of setting up new application and hooking github push requests to build new application instance.

Source code is hosted on github https://github.com/alexbeletsky/candidate. Don’t forget it is really early prototype, the code is in big mess now.. I’m going to improve it much soon, so please keep watching repository.

I had a lot of fun of building this and have a better vision of how the application should look like. Hope to share it soon. I would be happy to hear some feedback on this.. and of cause you are free to fork repo and make your improvement (especially in UI/UX :).

Hackatone in Kiev 11-12 June

This weekend I’ve been visiting very interesting event - Hackatone that took place in Ciklum office. Hackatone is whole night hacking session there every team or person could came up with any idea, implement it and present to crowd. Many times I heard about similar events either in podcasts or blogs that I read.. but never tried that myself. Thank’s for DOU enthusiasts I got opportunity to try that. Here is my report of that event. There would be some mix of English and Russian language, hope that would not make you confuse :)

The event began with common meeting there people were sharing ideas of applications they are going to build. During my pretty long journey with in Continues Production I already got my idea - I wanted to build Deployment-as-Service product that easily integrated with github and utilizes power of UppercuT for building .NET web applications. There was very few .NET developers there, already having their own ideas to build.. so I went alone.

If you have been to Ciklum office, you probably know what SkyPoint is.. It is 20-th floor in office that resides on one Kiev’s hills, so you might imagine this exceptional view from there. I grabbed a chair started to work.

Due to experience I got with .NET deployments and Jenkins I had pretty much well understanding what the workflow should be. I’ve created ASP.NET MVC3 in my Visual Studio and stared to develop. The scope of work I planned to do was pretty much big and I a little nervous about my ability to finish everything. I tried to work as fast as I can.. Full hour, that 10 minutes break and full hour again. Maybe because of that creative atmosphere around or straight orientation on final result but I think I really got a Flow..

So, after ~3 hour of work I’ve created my github repository for new project - Candidate.net:

tweet

Keeping that rhythm I continued to work mostly till 3.00 AM.. About 60% of work were done. I felt a little tired and decided to stop for today and had a good sleep to continue tomorrow. My plans were ruined as soon as I stepped to 19th there people gathered together.. So, in 5 minutes I got several shots of cognac and was sitting on a floor with guitar in my hands :). I had a real fun and relax there. Haven’t noticed at all how 2 hours are gone and sun started to rise.. We had a really nice moment of meeting new day on staying on office balcony.

tweet

I have to continue the implementation, so I went back to SkyPoint and run my Visual Studio again. Of cause I could not get such rhythm as I got yesterday, the process were much slower. I had to refuse of several more ideas I wanted to build, ignore the rules of Clean Code, SOLID and DRY. Did some ugly tricks, just to make application work. I had issues with testing and deployment since WiFi connection were quite slow. But, long story short - I implemented very basic workflow and realized that would be it for presentation.

tweet

I found a sofa and finally got a change to have a sleep. Don’t know why but I managed to sleep only for one hour, just woke up and could not sleep again. So, I had a shower (yes, there are showers in Ciklum office), coffee.. and very nice breakfast. I had some time before presentation began, so I re-tested my application several times more and was happy about results.

At 15.00 PM presentations began! I showed Candidate.net ability to initialize new application, build and deploy it in minutes.. How you can deliver with just a push a code to github, the rest of job done by Candidate. I don’t think I really impress people there, because of .NET crowd were to small.. but I tried to do best. After the presentation I’ve been contacted by several guys who interested with such concept.

Most of the presented projects were either old-school games implemented with new technologies or mobile applications. One guys did a nice linux kernel exploration and share his ideas.

After there was a manual voting for each project to decide on 3 best, I didn’t win :). Winners got their congratulations and presents for sponsors. I quit really quickly since I was a little late for my friend wedding that was that day :)

That was really nice experience for me. I appreciate Ciklum, DOU and everyone involved, who make that possible. Thank you guys :)

Inside ASP.NET MVC: Route to MvcHandler

In my previous post I promised to start with exploration of MvcHanlder as entry point of ASP.NET MVC application. MvcHandler plays the major role in MVC infrastructure, but as I fugured out - the whole story does not begin there. The story is actually begins inside ASP.NET framework.

ASP.NET framework

ASP.NET framework is heart of ASP.NET MVC application. Basically, ASP.NET is a request processing engine. It takes incoming request as input and sends it through internal pipeline till the end point. The architecture allows extensibility, you could plug either into pipeline (modules) or end points (handlers). Interesting fact that ASP.NET designed to be decoupled from actual request source. So, the source of request could be any application - IIS, Cassini or any custom one.

We could imagine that each request pass through the set of filters (internal pipeline) lands up on request handler (end point), request handler creates a response object and sent it back. There are a lot of hidden details of dealing with managed/unmanaged code that you could find in this brilliant article by Rick Strahl.

For us it is important to understand two basic abstractions, HttpModules and HttpHandlers.

HttpModules and HttpHandlers

From the developers point of view the difference between those are: one implements IHttpModule interface another implements IHttpHandler interface.

Module participates in the request processing of every request in order to change or add to it in some way.

Handler is responsible for handling the request and producing the response for specific content types.

ASP.NET MVC is HttpHandler.

MVC handler

There a bunch of default HttpModules and HttpHandlers in your IIS configuration. Just take a look here - %WINDOWS%\Microsoft.NET\Framework\v4.0.30319\Config\web.config.

But if you look closer you won’t notice any mention of System.Web.Mvc.MvcHanlder. Hey. what’s wrong? How the runtime actually knows that MvcHanler have to be called to handle our request? That was absolutely unclear to me at very beginning.

From request to handler

I opened WebRuntime.sln and found all references to MvcHandler. It is only one place there, in the MvcRouteHandler (MvcRouteHandler.cs) class. MvcRouteHandler implements IRouteHandler interface, with one single method GetHttpHandler. I put the breakpoint there and started application for debug.

MVC route handler

As application got started I halted on breakpoint with such call stack:

MVC debug

Let’s try to read and decrypt it.

It begins with WebDev.WebHost40.dll (Cassini Web Server) that receives request and System.Web.HttpRuntime.ProcessRequest. Here is ASP.NET framework start to work:

  1. ProcessRequestInternal method called. This is the primary method of whole pipeline. First, it initialize the HttpContext object.
  2. System.Web.Dll ProcessRequestInternal
  3. Next it get HttpApplication instance, initialize it and calls either BeginProcessRequest for IHttpAsyncHandler or ProcessReques for IHttpHandler.
  4. System.Web.dll GetApplicationInstance
  5. If I go to global.asax.cs of our application it is clear that System.Web.HttpApplication implementation is inherited and System.Web.HttpApplication is a usual IHttpAsyncHandler. So, async BeginProcessRequest starts.
  6. HttpApplication
  7. In BeginProcessRequest there is a cycle that iterates all modules and execute each of it. Reflector code looks messy, but the point is clear
  8. System.Web.dll ExecuteSteps
  9. And finally, the control goes to UrlRoutingModule that calls our factory method for MvcHandler. Aha!

UrlRouting module

UrlRouting is an HttpModule, registered in global web.config:

UrlRouting register

The goal of this module is basically matching incoming request by URL with pre-defined Route configuration and return corresponding handler for this request. In PostResolveRequestCache method it gets the RouteHandler and asks for corresponding HttpHanlder.

Wow, I didn’t know that!.. UrlRouting plays a major role inside the MVC infrastructure. But there the Routes are initialized and how MvcRouteHandler is associated with request?

Back to roots - Application_Start() method

I should actually start from here, Application_Start() is entry point of application. By my journey turn to be just other way around.

Application_Init()

Inside the RegisterRoutes we register route by giving: route name and constraints.

MVC map route

The MapRoute is actually extension method in RouteCollectionExtensions.cs.

MVC register route

Now, you can see how the route with particular URL is being register and associated with MvcRouteHandler.

Putting it all together

Now it is time to get deep breath and finally understand the route of HTTP request to MvcHttpHandler. Here we go:

  • ASP.NET framework starting to process incoming request in ProcessRequest method, but interesting stuff going in ProcessRequestInternal
  • ProcessRequestInternal calls GetApplicationInstance method that get application instance and calls it Application_Init()
  • Application_Init() registers routes that map URL (like “home/index”) with MvcRouteHandler instance
  • ASP.NET framework starts to iterate modules for incoming request and found UrlRouting module
  • UrlRouting module matches incoming request URL with registered collections of Routes
  • For found matched Route it gets corresponding route handler (MvcRouteHandler)
  • MvcHandler is being returned by GetHttpHandler of MvcRouteHandler

That’s the routing that is being executed each time you put URL and hit your browser button.

What’s next?

I hope next time we indeed look closer to MvcHandler :).

Previous: Inside ASP.NET MVC: Part 2: Setting up project for hacking Next: Inside ASP.NET MVC: All begins here - MvcHanlder

Inside ASP.NET MVC: Setting up project for hacking

As we already got debug assembly and pdb file, it it time to create a test project that would be a sandbox for all experiments. This would be classic MVC application, but we going to substitute release version of MVC framework with our locally built version, which allows us easily debug it.

Create new project

Let’s create new project. It would be MVC 3 Application:

new project

On a second step just select “Empty” in “Select template” dialog. “Razor” would be for view engine.

Correct system.web assemblies section

Go to web.config of a new project and comment out System.Web.Mvc and System.Web.WebPages assemblies. By doing that we saying: “those 2 assemblies are no longer a part of compilation of our application”.

web.config

UPDATE: If you do just that, later you will get an exception from Razor assemblies, about wrong type cast. To avoid that, it is even better to remove all assemblies in this section, plus Find/Replace PublicKeyToken=31bf3856ad364e35 to PublicKeyToken=null. That would do not make application to search GAC for missing assemblies and load locally built ones.

Correct application references

Now, remove existing references for System.Web.Mvc and System.Web.WebPages and put locally built ones instead. Right click on references project section and browse to the Debug folder of mvc3-rtm build directory. Try to build application now, it should be build without any issues.

setup

Home controller

Further steps are pretty simple, just add a test controller - HomeController with Index action. Add just a simple view for that.

controller

Check it out!

Place a breakpoint into action method and press F5 to start debugger. Now you we’ve got a full stack trace that allows to see every aspect of MVC world.

stack

Conclusion

The setup is easy and allows to jump in very quick. If you got any issues with that, please let me know.

What is next?

Next I will hack the entry point of MVC application called MvcHandler and understand how it works.

Please stay tuned.

Inside ASP.NET MVC: Packing bags for journey

I’m big fan of ASP.NET MVC framework. It is smart, clean and testable. A lot of stuff you can do with MVC are very intuitive, you don’t have to go thought pages of documentation to do some basic stuff. I do not use MVC on my primary work yet (unfortunately) but my small project became very nice sandbox to play around and learn stuff.

I definitely learned something thought the year of hooked up with MVC. But feel I still miss a lot of details. Moreover, my hackers curiosity always force to look - “what is inside?”.

Great stuff about ASP.NET MVC project is that is open source. It is been released as open source and remain open source project till version 3.0.. In fact, it is very easy to start learning framework from inside! This what I’m going to do. I would like to go depth and hope that journey would be interesting and valuable.

Today I’ll to prepare everything I need for journey. I would be happy if you join me, so we would help each other along the way :).

Getting out source code

That’s easy. Just google for “MVC3 source code” and it would bring you directly to ASP.NET Codeplex site.

Download mvc3-rtm-sources.zip. Unpack the archive and you will see the solution file WebRuntime.sln and 2 folders: mvc3 and webpages. Layout of both folders are similar:

    /--- bin
    /--- src
    /--- tests
    /--- tools

Source code package is pretty much organized. Solution does not using NuGet, btw :)

Meet the solution

It’s time open up solution! Fire up Visual Studio and open WebRuntime.sln. Let the journey begins!

Nothing fancy there. Minimal number of solution folders, just MVC and WebPages. Each of folder contains a bunch of projects in it. In nearest future we would be much interested in one System.Web.Mvc - heart of MVC framework.

solution

Build it

Don’t waste any time and just build the solution. As it usually happened to me, with OSS projects - you always have some issues during build. It is not the case for MVC. Build runs smoothly as possible. The output is being placed in mvc3/bin/debug folder. So, as soon as build finished check it out and see for content.

output

Run tests

All TDD lovers would be really pleased with a number of tests in solution. There is 5004 tests actually, solid number of tests. No surprise MsTest framework is used. Let’s just try to run them!

In menu Test -> Run -> Tests in Current Context. Here we go.. It took a bit time to run all. I was a little disappointed to see that some tests are failing.

I did an inspection of failures and found out that all of them are actually related to my Local Settings. I use “Ukrainian Locale” that means time format, decimal point symbol etc. are differs from “United States Local”. It’s a quite common issue of units, workaround could be a simply change your locale to the one that rest development team uses. The rests of tests run’s just fine!

tests

Conclusions

So, my initial impression are:

  • Easy to build and start up
  • Code looks really nice
  • Code style a little differs from default C# especially in with “{”
  • code
  • Tests following AAA pattern really nicely
  • test
  • Code contracts are not used, but code pretty defensive with check for null arguments etc.
  • exception
  • All members are with “_” underscore - I like that :)
  • Methods are small and clean

What I don’t like to far:

  • Solution is quite big - I would be happy to have separation between MVC part and WebPages (if it is possible)
  • Tests are really slow. Suite run for 3-4 minutes (upd: I blame MsTest runner for that, with TD.NET all tests passed for 89 sec’s)
  • Already found some comments that lie (one more point to try to avoid comments if possible)

What is next?

Next, I will setup a test project which allow us to fully get in deep of details and see how it work.

Please stay tuned.

.NET Saturday in Ciklum

This Saturday it was really nice event organized and hosted by Ciklum. I had a big pleasure to be part of the event and was given a speech about Continues Production (Delivery) and my experience of using UppercuT, RoundhouE and Jenkinks for that.

The organization of event was really nice. Only the speakers were one who introduced disorder there :). Some speeches were bit longer than planned, that’s why schedule ruined from the very beginning of event. My speech was planned for 0.5 hour, but I’ve been 1 hour on stage.

I’ve been pretty nervous since I never gave public speeches, but I really liked that experience. I hope to visit/participate more .NET events in future. At the end I was happy that everything is alright and I’m finally done. So, in great mood and relaxed I stayed to listen for the rest of speakers.

Also I was really nice to meet Sergey Kalinets, Anton Vidishchev who shared their experience in TDD and ASP.net MVC framework. Such type of event are really great for professionals gathering, helping to better communication and knowledge sharing.

Let me wish Ciklum and Inna Tueva in particular to have will and energy for organizing such events. Thanks a lot!

Continuous Delivery: Make it work

As soon as we setup project infrastructure in a way that build and deploy is done by one batch command, we are ready to run this process continuously. It is possible to take any CI server you want, I stopped on Jenkins.. basically it contains everything I need. But before start with Jenkins, let’s one more time take a look current infrastructure and goals.

Git branching model

There is great article on that topic, I used main ideas from that. So, I have 2 branches in my origin that exists all the time. They are: master and development. All pushes I do during implementation are going to development branch. No direct pushes are made to master at all (there are some exceptions of cause, but I try to follow that rule). As soon as code in development is stabilized, I prepare special release branch (update version and small clean up there) and release is being merged to master as it tested and everything is fine.

Staging and Production environments

I define 2 types of environments: Staging and Production. Environment includes: binaries, markup, database, deploy scripts.

Everything that is being developed are goes to staging environment immediately (and automatically). Production update is being run manually as soon as all testing on Staging are finished. Separation of environments is great idea. Even if we are trying hard to have potentially shippable software during each build, this is not true. It is usually a lot of problems in latest version that need to be fixed before production server update.

So, Staging is a result of build/deploy of development; Production is a result of build/deploy of master.

In my case, everything you see here is a result of build of this. Everything you see here is a result of this.

What I want to get?

I want to have automatic system that would update my Staging with every push to development branch and I want automatic system that would update my Production as soon as master branch is ready to. As it said in overview having such system is great reduce of deployment overhead. Let’s make this happen!

As I said earlier I would use Jenkins. Great stuff about this software that is very intuitive! Setup is easy and fun, I would put just a little guidance of process.

Local Jenkins server setup

As you download Jenkins WAR file, it is easy to start just with java -jar jenkins.war. Now, the dashboard is available at http://localhost:8080/.

Plugins

First, you should add all required plugins. It already contains some, but some are missing. I’ve add such to my default configuration:

Go to “Manage Jenkins” -> “Manage Plugins” and install missing one.

So, my configuration looks like this:

plugins

Configure system

Now go to “Manage Jenkins” -> “Configure System” and find “Git” section there. You should provide with your user name and path to Git. I have git in my PATH variable (recommend you do the same).

Create new job

After basic configuration is done, let’s define new job. Go to “New job”. In common section you should give your job name and provide with githib path to repository.

common section

In Source Control Management: URL to Git repository, Branch to build and Repository browser (optional).

SCM

Build triggers (explain a bit later).

build-triggers

In Build section you specify commands to build/test/package and to deploy results. See my previous post for details.

build

Finally in Post-build section, specify path to grab artifacts.

Test it locally

As soon as configuration is done, you can test your build locally. Just start your job and make sure that: build run, tests executed, new database deployed, new application deployed. Basically if everything done correctly in previous steps you should not experience any troubles. If it is fine that means the configuration is simply ready to be put to production server.

Move it to production

As it defined, our goal is to update staging and production with out any manual work. The simplest scenario then CI server resides on the same machine as production server.

Great thing about Jenkins is that it requires no installation. So, I packed all content of ~/User/Jenkins folder and Git into one single package and FTP’ed to my production server. As you unpack, configure Git in PATH variable, setup up your private repository keys it is just simply ready to work. Now, test it remotely - staging and production should be able to update with one single click on Jenkins dashboard.

Little adjustments for production server

My VPS has Windows auto-update enabled, so It is being rebooted from time to time. As it is being rebooted, you should start Jenkins manually. Fortunately it has really nice feature - Jenkins might be run as Windows Service. This is extremely useful, I recommend to do it from very beginning.

Using Github hook to automate build run

I want my every commit to development to be tested and to be visible on Stage. As it said in Configure system part, you can setup build triggers. With build trigger you can run build just by requesting this URL http://jenkins.server/jobname/build?token=TOKEN.

In github repository go to Admin section and setup new hook in “Post Recieve URLs” section, like this:

github

This concludes the setup.

Control your builds all the time

What we did is only very basic setup, but it already gives a lot of power. It is really easy to track all builds/artifacts/history of changes. Downtime of site is extreamly slow (comparing to manual changes). Dashboard is very convenient to control builds:

dashboard