Alexander Beletsky's development blog

My profession is engineering

ASP.NET MVC Routes and Namespaces

While I was working on ELMAH.MVC v.2.0.0 I noticed a something that contradicts the way I understand how the controller resolving mechanism works. Before, I always thought that namespaces matters, but in practice I saw it otherwise.

Controller in separate class assembly

The good way of distribution re-usable software is class assembly. Suppose, I have 2 projects - one ASP.NET MVC web application (MvcApplication2), and another one is class assembly (Awesome.Mvc.Lib). Web application references the class library.




I want to have some particular controller to be exposed from Awesome.Mvc.Lib. Namely, I want to have a controller inside the class library, that would be accessible from MvcApplication2. I’ll add some ShinnyController.cs inside.

namespace Awesome.Mvc.Lib
{
    public class ShinnyController : Controller
    {
        [HttpGet]
        public string Index()
        {
            return "I'm in Shinny controller";
        }
    }
}

Originally, my thought was, ShinnyController will always be “invisible” for MvcApplication2, since it placed into another namespace. Meaning, if I don’t initialize a route pointing to ShinnyController, the routing mechanisms would never match it. But, if I run the application and go http://localhost:26810/shinny I will see:




This is totally unexpected to me! It turns out that default route is matching the controller/action from Awesome.Mvc.Lib class library.

routes.MapRoute(
    "Default", // Route name
    "{controller}/{action}/{id}", // URL with parameters
    new { controller = "Home", action = "Index", id = UrlParameter.Optional } // Parameter defaults
);

What I’m expecting though is that ShinnyController.cs have to be “explicitly” routed, and ideally placed into its own sub-URL, like http://localhost:26810/awesome/shinny.

What about namespaces fallback?

I’ve asked this question on stackoverflow. Even if I had good answer, it did not make it happy. So, to get the behavior I want, I need to do the following:

  1. Change the default routing to explicitly mention the namespace and set fallback to false:
  2. public class MvcApplication : System.Web.HttpApplication
    {
        public static void RegisterRoutes(RouteCollection routes)
        {
            routes.IgnoreRoute("{resource}.axd/{*pathInfo}");
    
            var route = routes.MapRoute(
                "Default", // Route name
                "{controller}/{action}/{id}", // URL with parameters
                new { controller = "Home", action = "Index", id = UrlParameter.Optional }, // Parameter defaults
                new[] { "MvcApplication2" }
            );
    
            route.DataTokens["UseNamespaceFallback"] = false;
    
        }
    
  3. Create an Area in Awesome.Mvc.Lib and configure routing to it:
  4. public class AwesomeAreaRegistration : AreaRegistration
    {
        public override void RegisterArea(AreaRegistrationContext context)
        {
            context.MapRoute("Awesome_default", "Awesome/{controller}/{action}", new { action = "Index" });
        }
    
        public override string AreaName
        {
            get { return "Awesome"; }
        }
    }
    

After I did so, I can reach the http://localhost:26810/awesome/shinny:




And in the same time, http://localhost:26810/shinny is getting to be rejected:




Even though, it looks like desired behavior.. It sucks.

Why it sucks?

By placing the controllers into separate assembly, I’m thinking about it’s distribution by simple bin-deployment or by Nuget. Both ways assumes, simple copy of assembly into particular location, adding references and that’s it! I don’t suppose to change default routing that comes in ASP.NET MVC applications templates.

I want to have the control of routes *inside* the class library, not outside of it (in web application). But, the default behavior of ASP.NET MVC routing is completely different. Moreover, in some cases I want users of library to be able to control the routing.

In my opinion the default behavior with UseNamespaceFallback = true is wrong. I’ll give one more example, to proof it.

I removed namespace fallback code from default route, after added new Area, called Api. Inside this Area I place one controller, called SimpleController.

namespace MvcApplication2.Areas.Api.Controllers
{
    public class SimpleController : Controller
    {
        public string Index()
        {
            return "I'm simple controller from API area";
        }

    }
}

The controller is reachable, as expected:




But now, I try to access /simple:




Hey WTF? The whole idea of Areas is just ruined. What I’m doing wrong?

I’m feeling very frustrated about this issue. Even though I understand why it happens, it smells like a bug for me? It works exactly the same for ASP.NET MVC 2, 3, 4. I’m asking you guys, to help to clarify the problem. What is your opinion on that? Are you agree on such default behavior?

Three month without .NET code

It’s not farewell letter in style of “Goodbye .NET”. But, I just realized that I haven’t created any .NET code in about three month. That is pretty much. Mostly all the stuff I currently do, both at work and for fun projects, I do in JavaScript.

I love .NET and love C# language. C# is probably one of the best language I ever programmed with. I think that .NET is my comfort zone. Web technologies as ASP.NET MVC, Web API, NancyFX are great. But, as I said long time ago server side development is not enough to create cool web apps.

Front-end development

For modern web application, the front-end part became as equally important as back-end. In e-conomic we are doing large scale JavaScript applications. According to Addy Osmani’s definition, the large scale JavaScript applications are:

non-trivial applications requiring significant developer effort to maintain, where most heavy lifting of data manipulation and display falls to the browser.

This is exactly definition of application we released just few days ago, debitoor.de. And if you are doing front-end, the JavaScript is your friend. Actually, you don’t even have a choice.. It’s either JavaScript or some another language that could be compiled down to JavaScript as CoffeeScript, ClojureScript or Dart languages. The importance of JavaScript, it’s common adoption, growing of community knowledge base made Scott Hanselman call JavaScript assembly language for web.

We’ve chosen Backbone as front end development framework. With a backbone JavaScript code became to have structure. Having a proper code structure is vital for any project. I’ve started to learn Backbone and even I still thinking it’s too minimalistic and non-complete, I believe that was a huge step forward.

Having implemented and stabilized HTTP API for application, almost all new features implementation was at front end. That means, new features for product were actually born in JavaScript code.

Back-end development

As might know from my earlier post we’ve chosen ASP.NET MVC to build our HTTP API on. That worked well at the beginning, but later decision were taken to move out the API to Node.js platform. So, another team started to work on new API project, basically implementing existing API interfaces from ASP.NET MVC in Node.js.

Being wisely architected, our front-end did not have major ASP.NET MVC dependencies. We did not use any specific features (like Razor) or anything else just in sake of minification of efforts during switching back-end platform. And I would say it worked really well.

Node.js based API built on Express.js framework, that suites nice for such kind of work. There is a great blog post about convincing you boss to use Node.js. Node.js + MondoDB is very suitable for JSON API applications, for several reasons. First of all is performance, Node.js is known for it’s async I/O operations that is so perfect for web. Second is simplicity (relational simplicity, of course). Serving JSON objects inside JavaScript, storing/restoring them as MongoDB documents makes the platform and approaches very solid.

So, the new stuff from HTTP API also got born of JavaScript code. Check out nice post of Allan Ebdrup (one of the Node.js inspiration guys inside e-conomic) about our experience with Node.js and MongoDB.

How does it feel like?

So, I’ve started to work almost all the time for front-end having Sublime Text 2 + Chrome Developers Tools as IDE. It was a really unusual at the beginning, after many years in Visual Studio.

I feel myself more productive with JavaScript now. I write the code with JavaScript construction and sometimes thinking myself “how would I do in C#?” and it feels like in C# it would be much harder. I like dynamic nature of language.

Besides the coding I have pleasure of new infrastructure. Setting up .NET solution, containing ~80 projects, long build process, setup of MS SQL and re-staring of IIS.. All of that made a lot of idle time, read wasted time. I’m in idle, during waiting till build is finished.. or waiting 30 sec while IIS update AppDomain if I rebuild and press F5. It easily can take up to 2 hours per working day just wasted.

Node.js + MongoBD environment is extremely easy to setup, amazingly fast to work with. I’m restaring the application in 1 sec, just by stopping node process and running it again. With a tools like nodemon, you don’t even need to do that. Just save the code and after a sec, press F5 in browser to test it. All of that are increases general developers happiness.

JavaScript is difficult language and Node.js is new platform that requires respect and time to learn. Node.js is probably one of the best documented projects I ever worked with. Besides, it’s open source. It means you can learn from sources. I haven’t dig inside Node.js code itself, but I did look inside the Express.js then I had an issue and I found the answer as I see how particular method works.

Even though, Backbone.js code and Node.js code are completely different stuff, having the same language both front-end and back-end is great thing. In particular it minimizes the language context-switching then you do coding. It’s possible to apply similar patterns and practices both ends.

Conclusions

Being a JavaScript developer works good for me so far. I opened just entire new world and would like to explore it more. I hope JavaScript would be my comfort zone some day.

Starting Up Node.js Development on Windows

Node.js is very interesting platform, it allows to unleash your internal hacker and gives a chance to join one of the fast growing developers communities in the world. For a quite long time, node.js was available only for Linux and Mac, but due to latest Microsoft contribution to project Windows is supported in a very fine way.

If you are .NET developer addicted to Visual Studio, you might feel a little frustration about joining new camp. Primarily, because of lacking proper development environment. This is not quite true. You can setup very solid development environment for Node.js on Windows box.

Installing node.js

Node.js is nothing more as node.exe file. It’s very easy to install with latest installer available on nodejs.org web site.

Installing the software is always a little mess. You have to download, un-package, run.. go different wizards - it’s boring! Do you remember how you life changed after NuGet release? What if say, you can do exactly the same thing, but instead of installing .NET packages, you can install software. This is possible by brilliant project called Chocolatey.

After Chocolatey is setup, you can easily install any package you like, including one for node.js.

c:> cinst nodejs

Nice and easy.

Trying things out

As Chocolatey installed node, it should be available in your PATH. So, fire up new cmd window and put ‘node’ in command line. node.exe acts as javascript interpreter, so you can directly put some code in console. Here is simple example of Fibonacci number’s calculation in node.


node.js command line

Instead of typing the code into interpreter console, you can save it to local file ‘fib.js’ and run it with node.


node.js command line 2

Setting up development environment

Basically all you need with node.js is command line and text editor. Any text editor you like. You just code in you favorite one and then switch to command line to start up application, like ‘node app.js’.

You can go further this minimalistic setup, thought. I’ll show the best options I’ve tried so far.

Sublime Text 2

Sublime Text 2 is a superior development-oriented text editor. Even with plain installation it’s very useful. Available on many platforms, including Windows makes it really ‘must-have’ tool for any developer.

You can turn Sublime Text 2 into quite powerful node.js IDE with plug called SublimeText-NodeJs.

It provides with basic code completion functionality.


SublimeText-NodeJs

But it also gives you ability to simply run node.js scripts inside the editor and see resulted output.


SublimeText-NodeJs

Just inside the editor you able to access ‘npm’ and ‘node’ command line, so it’s pretty possible to work with project without leaving text Sublime at all. Reality is a little different, thought.

You can handle pretty big projects inside Sublime. It is able to open folders, providing great file navigation options. Pressing ‘Ctrl+P’ will bring navigation console. Start typing the file name and relevant suggestion will appear.


SublimeText-NodeJs

As well, you can easily navigate inside javascript file, by functions or object properties. Press ‘Ctrl+P’ after put ‘@’ sign and type function name.


SublimeText-NodeJs

JetBrains Web Storm

JetBrains produces great products including Web Storm. Web Storm is full feature IDE from a world-class vendor. Being HTML/CSS/JS editor, it could be easily adopted for Node.js development.


JetBrains WebStorm - nodejs

With a WebStorm you just create a new project, place javascript files inside and configure IDE to run Node.js configuration. You have ready to use presets, including one for node.


JetBrains WebStorm - nodejs

As you start to work with any Node.js internal modules (as http, fs etc.) WebStorm would propose to download sources and index them to be able to produce IntelliSence-like suggestions. I was really happy to see how fast and precise it works.


JetBrains WebStorm - nodejs

Besides of that, you are getting all JetBrains IDE’s goodness - Refactoring, Integration with VCS, Code Formatting, Navigation etc. But the most sweet feature as for me is - Debugging. Just place the break point in the line you want to halt and press ‘Debug’ button.


JetBrains WebStorm - nodejs, debugging

App would run and debugger will stop on breakpoint, showing current Frames, Local or Watched variables.


JetBrains WebStorm - nodejs, debugging

Microsoft Web Matrix 2

New kid in the block of Node.js development on Windows is Web Matrix 2, currently beta version.


Welcome to Web Matrix 2

Web Matrix 2 is a lightweight IDE suitable for ASP.NET, PHP, HTML development. It’s easy to install it by Web Installer. As soon as it’s installed, you can pretty nice templates for Node.js applications.


Welcome to Web Matrix 2

Web Matrix works differently. As Sublime and WebStorm depend on ‘node.exe’ and work directly with it, Web Matrix uses another component, called iisnode. IISNode is a host of node.js on IIS web server. This approach of course brings a lot of benefits, but in the same way it might give you some limitations of IIS itself. Web Matrix comes with IIS Express web server, which is perfect for web development. One very obvious benefit of using ‘iisnode’ is that you no longer need to restart ‘node.exe’ process manually, as you changing sources.

But let’s go back to IDE itself. As for IntelliSence it much more worse that WebStorm and even Sublime, I would say. It works really strange, for instance knowing nothing about embedded node components.


Welcome to Web Matrix 2

No internal debugger, no refactoring tools. The code editor is more than good enough. Almost the same user experience as Visual Studio.

Besides of the code editor, you create databases and different report for sites.


Welcome to Web Matrix 2

Conclusions

Even with plugins, Sublime Text 2 still more text editor than IDE. Definitely, more than good enough text editor, but still. I personally using it. First of all, because I like.. second, because I would like to be closer to ‘node.exe’ at least at the period I’m learning the platform. Available as unlimited trial for now, makes it really attractive.

WebStorm, no doubt is the most valuable player now. It works great, even If I heard some rumors that it sucks a little on Linux. This should be the choice for professional development. It costs money.

Web Matrix 2 is in beta, but even now it looks pretty solid. It goes free and I suppose it will remain for free. It’s much more closer to Microsoft technologies, like IIS, Azure etc.

So, what to pick up you might ask?

If you are just starting and learning and really want to keep similar experience to VS, pick up Web Matrix. Otherwise, it have to be Sublime Text 2. Both perfect fit for home projects. In case if you do professional development on Node.js, obvious choice is Web Storm.

Using Approval Testing for ELMAH.MVC project

During the ELMAH.MVC 2.0 preparation I had a bit challenging task. As I mentioned, I was about to adopt some code from ELMAH.Sandox project. The 2.0 release included some new features, but what was important to me is make sure I keep previous ELMAH.MVC functionality.

In case you are following my blog, you probably read about Approval.Test framework, which I personally like and recommend to my fellow developers. I did some posts, there I tried to show some Approvals benefits. Today, I’ll show some real-life use case.

Installing by NuGet

Last time I played with Approvals, the binaries were only available at SourseForge project page. It is not convenient at all. I was happy to see the tweet from Llewellyn Falco, he mentioned that ApprovalTests are now available on NuGet. So, installation now as easy as:

PM> Install-Package ApprovalTests

Thinking of test

I had to have some kind of assurance, that I will not break existing ELMAH pages after I switch to new ELMAH controller. Basically, I need to have some ‘Master Database’ that would contain all logged errors and I need to grab all possible output that is being generated by ELMAH error log page handler.

This is what perfectly match the ‘Locking Down’ testing strategy. In locking down testing, you try to get all possible system output, and approve it. In my case, I wanted to approve that all HTTP calls to ELMAH pages, like ‘/elmah’, ‘/elmah/stylesheet’, ‘/elmah/rss’ and so on, still work as they worked before, meaning producing the same output as they produced before.

Test implementation

For my ‘do’ step of the the test, I need to collect all possible applications output. I already know all URL’s, so I just fired up the site with ELMAH.MVC 1.3.2 version installed. I configured the site to use XML files for storing the tests, so I can easily copy them before each test run. That made a kind of ‘Master Database’ for me.

For the verification, all I need to have is ApprovalTests.Approvals.Verify().

[UseReporter(typeof(DiffReporter))]
public class ElmahMvcTests
{
 private const string ElmahMvcAppUrl = "http://localhost:49800/elmah";

 [Test]
 public void lock_elmah_mvc_pages()
 {
  // do
  var content = new StringBuilder();            
  var pages = new[] 
      {
       ElmahMvcAppUrl,
       ElmahMvcAppUrl + "/",
       ElmahMvcAppUrl + "/stylesheet",
       ElmahMvcAppUrl + "/rss",
       ElmahMvcAppUrl + "/digestrss",
       ElmahMvcAppUrl + "/detail?id=5dd2a560-c6fd-4847-a6cc-e3e253db5764",
       ElmahMvcAppUrl + "/json?id=5dd2a560-c6fd-4847-a6cc-e3e253db5764",
       ElmahMvcAppUrl + "/xml?id=5dd2a560-c6fd-4847-a6cc-e3e253db5764"
      };

  foreach (var page in pages)
  {
   content.Append(GetContent(page));
  }

  // verify
  ApprovalTests.Approvals.Verify(content.ToString());
 }

The GetContent() is responsible for getting page content. It could be as simple as ApprovalTests.Asp.AspApprovals.GetUrlContents(url), but you might get in the trap, as I did so.

Dealing dynamic data

The trap is called ‘Dynamic Data’. Dynamic data is every dynamic part of you output. In my case, the dynamic part was a server time generated by ELMAH and putting into page footer. Of cause, even if I run the test and approve the results, on a next test run I still have red test, since I have differences in a seconds of timestamps in footer. It’s annoying, but nothing you can do about it.

My solution was simple. Since I actually don’t care about those timestamps at all, I can simple cut it away from output. The rest of the document, including structure and content would remain the same, so testing quality will not be lower.

So, the GetContent() method will look as follows,

private string GetContent(string url)
{
 return RemoveFooter(ApprovalTests.Asp.AspApprovals.GetUrlContents(url));
}

private string RemoveFooter(string content)
{
 var pattern = "<p id=\"Footer\">(.*)</p>";
 return new Regex(pattern).Replace(content, string.Empty);
}

Approving the results

After I had stable test output, I safely approved that test. Approving in terms of ApprovalTest framework, is simple copy of Accepted file into Approved file.

Now the system got into ‘Locked’ state and I’m safe to do the changes. This is something as Llewellyn mentioned 100% test coverage with one test. Indeed, having only one test and spent 10 minutes to create it I covered huge piece of functionality.

Keep that safe

After I applied all my changes, I re-run approval test to see how it works. I was happy to see, that I’m still generating the same output as it was before. It means, the system still functions as it’s expected. All routes works, generated content does not have differences inside. Cool!

As I said earlier, you don’t need to keep the test for all time. In my case, as I completed my refactoring and saw that code still works (and of cause, manually tested the application), it’s OK just to delete approval test. No longer needed.

Approval.Tests are really shines for a scenarios like that.

CS101 Course Accomplished!

Yesterday, I have finally received results for my Final Exams of CS101 Udacity course. I did it!

I have successfully accomplished that course, received the certificate with “High Distinction”. To be honest, I thought it would be “Highest Distinction”, since I was pretty sure about correctness of all my solutions posted. But reality is different, you can never be sure. Next time, I’ll probably put more attention to verification of my code. Nevertheless, here is my certificate.


certificate

I’m really happy that I managed to make this happen. It’s probably nothing to proud for, but I feel much motivate for further Udacity courses. As I said in previous post, I enrolled for next ones on statistics and algorithms. Hope I will be able to go through them, with at least same results I got for CS101.

Once again, thanks to Udacity for such great opportunities.

ELMAH.MVC v.2.0.0 - Release Candidate

I’ve just pushed new Elmah.MVC package. It contains several changes that I described on my previous post. Current version is v.2.0.0-rc, release candidate that I’m about to spent sometime with testing and collecting feedback before it’s availability.

I think ELMAH.MVC is getting it’s shape. Hopefully it will be as easy to use as indented.

Changes highlights

Quickly recap the major and breaking changes of ELMAH.MVC 2.0.

Deployed as class library

First noticeable change is that ELMAH.MVC is no longer deployed as several source files, having new Area and Elmah controller inside. The the sake of compatibility and reach functionality it is now deployed as Elmah.Mvc.dll. It’s using David’s Ebbo WebActivator project to bootstrap itself, registering routes and global filters.

New URL to access ELMAH dashboard

In 2.0 ELMAH is accessed by /elmah route (not /admin/elmah as it was in previous versions). There are several reasons for that. First one is because of new ‘Admin’ area is not created anymore, second is the really strange behavior is ASP.NET MVC default routing rules.

HandleErrorAttribute inside

If you tried to use ELMAH in ASP.NET MVC, you are probably implemented your own HandleErrorAttribute, as it’s shown in this example. You no longer need to apply this custom code with Elmah.MVC. As soon you installed package, so can safely remove your HandleError attribute, since it’s already included into package.

No more /elmah.axd

In previous Elmah.MVC version, you have to go to web.config file and remove all handlers manually. Elmah.MVC 2.0 is now having elmah.corelibrary as dependency and provides it’s own web.config.transform that excludes all legacy stuff. In short, no more /elmah.axd to remove.

Configurable authorization rules

Now, you are very flexible with authorization rules of ElmahController. web.config contains several elmah.mvc.* app keys you can tweak it. Disable or apply authentication based on roles, no problem.

Logging with customError=”On”

Something that Troy Hunt noticed and registered. Now, you should not have any problems with logging unhandled exceptions, even if customErrors is enabled.

Trying it out!

I would be really happy if you help me to test it. You are welcome to try it out. Please install new package and let me know your opinion or bugs you might find. To install preliminary version, just do.

PM> Install-Package Elmah.MVC -Pre

ELMAH.MVC v2.0 is coming

Elmah.MVC is rather popular package in NuGet gallery. Being submitted about year ago it currently has about ~14K downloads. I received a lot of nice feedback and to be honest feel quite happy that people are using it. Being extremely simple it requires almost no effort to support.

Anyways, several month ago I received an email from James Driscoll one of the major contributors to ELMAH project. James says that he is working on similar problem, namely easy to pickup ELMAH error handling within ASP.NET MVC applications. He saw my package, but in the same time he has his own prototype in ELMAH.Sandbox project, hosted on google-code. Instead of creating yet another NuGet package he proposed to join our efforts and release next version of ELMAH.MVC. That was the beginning of ELMAH.MVC v2.0.

What will be changed?

Elmah.MVC is really simple one, actually delivered as 2 C# code files. It creates new ASP.NET MVC area and register route to ElmahController, which is just an adapter to ELMAH LogFactory. It worked very smooth, but with more functionality this approach no longer work.

Pack ELMAH.MVC into separate class library

The idea is to bother user as less as possible. Ideally he just drops the DLL into Bin folder and it start to work. In real world it’s a bit more complex, but anyway. Elmah.MVC will be packed and delivered as class library.

Support in VB.NET projects

Since it will be just a class library, VB.NET developers finally could use Elmah.MVC without any manual code conversion. I don’t know how many ASP.NET MVC3 applications are developed in VB.NET, but I think it’s still nice step towards compatibility.

HandleError attribute

A lot of people, including myself are using HandleErrorAttribute on controllers to redirect user for nice looking error page if something goes wrong. This mean you handle the error by yourself, so it won’t appear on ELMAH log. That is very big disadvantage of current Elmah.MVC. This will be fixed as new global filter added, that will care that everything will be properly logged.

Several small fixes

So far, I’ve collected some issues that just was in line and waited to be fixed. It’s a good time to get rid of those.

What’s the plan?

Good for me, James did all heavylifting. I will backport some good code into new project. I’ve created new branch and already started to work. Unfortunately, I could see some breaking changes there. At the end of this week, I’ll have new package ready to test. You can track the progress on github by watching v.2.0 milestone.

I will need your help guys to test that thing out. If everything goes fine, I’ll update NuGet package as soon as possible.

CS101 Building a Search Engine: Week 7 and Final Exams

As I said in my previous post, last several units of CS101 were a little tought. Not so difficult, but it made me a little worry what’s it gonna be in final exams. Being posted right in time, I had to conclude Unit 7 before I start with exams. That time, I had some doubts if I’m able to meet course deadline.

Fortunatelly, Unit 7 was non-technical one. I went throught it rather quickly. Some scenes for this unit were taken in Compute History Museum. This place is just amazing, I wish I had a chance to visit it someday. We’ve been passed throught a pre-historical computers (like Babbage machine), first hard drives and other interesting stuff. Besides of that we guested Mozilla company talking to it’s president and developers about open source projects, infuence of open source and some thoughts on how to be involved in open source community.

The final exam appeared to be not so difficult. I submitted it in time, even earlier than expected. I was also quite surpised, to see that deadline is a little prolonged up to 4th of June. Anyway, the solutions are submitted and now it would take up to 2 weeks waiting for results. I really looking forward and hoping to get my certificate.

Conluding this series of posts I have to say - Udacity courses are awesome! I really appreciate David Evans and Sebastian Thrun for making all that happen. My big credit goes to David who had lead CS101 and was just a perfect professor, clearly describe all the material and showing nice examples. I would like to say ‘Thank you’ for everyone envolved.

I think CS101 was great ‘warm-up’ course. I would not stop on that, but I already enrolled for next ones. This time I’ve choosen more fundametal - ST101 - Introduction to Statistics and CS215 - Algorithms, Crunching Social Networks. Both courses are starting June 25. Not sure, would it be possible to make them on parallel, so I might hold one if it would be to difficult. Besides of that I’m currenty looking throught CS253 - Web Application Engineering with Steve Huffman (the creator of reddit.com) just as a free course, to play a little more with Python and Google App Engine.

I highly recomend you to pickup some course today. And once again, thanks a lot to Udacity for that priceless opportunity to learn!

Is TDD Required Skill? - Two Years Later

TDD

Two years ago I’ve concluded an interesting research. I was thinking how much employers care about hiring developers with TDD skills while looking for suitable candidates. As output of this research I’ve created a simple crawler, that is able to crawl vacancy web sites and perform the simple analysis does this vacancy description require TDD skill or not. As well, I’ve created some blog posts, where I did share results + some technical issues I met during building this crawler.

At the weekend, I’ve decided to repeat this experiment. The code of crawler worked perfectly, except some correction I had to apply to conform the markup changes of sites that happed through these two years. Leaving the technical part of this blog post, I want to share the results with you.

What was at 2010?

Let’s recap what it was at 2010. I’ve crawled 978 records from rabota.ua site and 150 of 978 (about 15%) contained TDD requirement. For careers.stackoverflow.com the results was a little better, ~23% (49 of 212) contained some mention on TDD.

The quote from 2 year ago report:

.. such demand on TDD could be reasonable for countries like Ukraine, where major market is for maintenance of existing code base (typically legacy code, no tests). But we see similar figures for USA, country where a lot of brand new product born.

I believe the awareness of TDD and general software quality got higher along the way. But let’s see the figures of 2012.

What we have in 2012?

This time I got a litle more data. 1067 records of rabota.ua and 560 records of careers.stackoverflow.com. But what about demand?

All right, it’s only 77 vacancies of 1067 is requiring TDD, which is 7.22% in rabota.ua. careers.stackoverflow.com has 127 vacancies requiring TDD which is 22.68%.


The technologies breakdown is follows. Java leads as a technology with the highest demand on TDD. During this post creation I thought that my results does include JavaScript, which gain much popularity recently. It’s interesting to see what’s there.


Conclusions

Ukrainian market demand lowered almost twice. USA market demand is stable, with a little change which is almost the noise.

  • Ukraine: 2010 -> 15.34%, 2012 -> 7.22% ( -8.12% )
  • USA: 2010 -> 23.11%, 2012 -> 22.68% ( -0.43% )

I can see very low demand on TDD in C/C++ field. Almost equal demand between Java and .NET with a slight win of Java.

I personally wanted to see different figures. Anyway, it was interesting to see actual results and restored some code that worked 2 years ago. Probably this analysis required to be done more frequently to have a kind of valuable statistics.

7 Weeks With Python

During my CS101 class I first time meet Python. That’s not completely true, since I played a bit Python before, but I had a chance to see Python in action only during this course. In short, that was really pleasant meet up and I hope it will grown up to prolonged and mutual relationships. I will share some ‘likes’ and ‘dislikes’ about Python, which I would think you find interesting.

What I liked in Python…

Python is very well known language with great reputation and community around it. Big companies, including Google, using python to build enterprise level applications. Hackers love Python cause it combines both simplicity and power. You can do any kind of application development including desktop and web applications. All the time is being compared to Ruby, which is for me ends up to something like this only.

No IDE development

You don’t need any kind of fancy IDE to start up with Python. Sure, IDE is something that makes development more efficient, so if you going to do a lot of programming with Python including debugging you should probably pick one. But for now, I’m totally happy with Sublime Text 2 as IDE.

Sublime Text 2 for Python

Easy to learn

If you know some OO language like C++ or Java, it will be quick jump to Python. Python is object-oriented language but with support of different paradigms as procedural and even functional programming. The basic concepts of variables, conditions and control flow are the same as you get used to. Of course, you spend sometime to know the fundaments - like, how to compare things, how to calculate length of string or list, how to put element into dictionary. Sometimes, I still refer to documentation, but in general all that things are easy to remember with practice.

Interpretation and dynamic typing

Python is interpretator. You never mention the type of object as you declare it or use it. You might apply different operations on object, it would be evaluated on runtime. There are different opinions (holy wars) on Static vs. Dynamic, but as for me with Dynamic languages the overall development velocity is higher. First of all, you don’t spend any time for compilation, which in case of big solutions could be really high. Second, as you are only to able to check results as code executed (even if you just misspell variable name you will know about it only if this code section is evaluated), you are more focused on unit tests and TDD to catch up obvious issues, which in general makes development faster.

Built in types

Python has complete built-in types system. For numbers you can use different types, as int, float, long, complex. The type is evaluated on runtime,

            i = 0               # integer
            j = 1.0             # float
            x = complex(1, -1)  # complex
        

Strings are everything inside the ” quotes,

            str = "I'm string"
        

Btw, during the course I got conclusion that list is the most flexible data structure. Everything you need, you can build upon lists. Lists are nicely handled with Python,

            l = [ 1, 2, 'three', [4, 5], [[6, 7], 8]   
        

Lists are non-optimal for searches, so if you do a lot of searches you might consider using dictionary,

            d = { 'one': 1, 'two': 2, 'three': [3] }
        

Each type has it’s own set of methods. Strings including common operations as concatenation, trimming, substring found etc. For list and dictionaries there are bunch of useful stuff as getting iterators, pushing and poping of elements.

Syntax and Code styles

Syntax and Code styles are commonly another topic of holy war. Fortunately, Python leaves very few room for that. First of all - no semicolons. Second, Python uses indentation as part of language syntax. So, poorly indented code would simply won’t work.

            def my_method_name(arg1, arg2):
                return arg1 < arg2              # right
                
            def my_method_name(arg1, arg2):
            return arg1 < arg                   # won't work!
            
            def my_method(arg1, arg2):
                if arg1 < arg2:
                    return arg1 + arg2          # right
            
                if arg1 == arg2:
                print arg2                      # won't work!
        

Basically, everything that is after “:” have to be indented. This is for if, while, for etc. Using tab instead of curvy braces (or any other symbol or term) is really nice idea, allowing to keep code in the same format and ‘forcing’ one and solid code guidelines along the projects and teams.

What I disliked in Python…

There are no perfect things, moreover perfect languages. Developers have their own habits and opinions on different things. In Python I see several things that makes me little uncomfortable about.

Naming consistency

It sounds like Python adopts C code styles for naming of methods and variables, like longes_cycle or def make_it_work(). But in reality a lot of methods are violating those rules. For instance some methods of dictionary type: fromkeys(), iteritems(), setdefault(). In the same time dict contains the method: has_key().

That’s very annoying. Especially if you don’t have any IDE with names suggesting, it makes it really hard to remember.

Booleans and None

Almost the same as in topic above. Having the C-style (with a lower first symbol) language designers decided to have a special cases.

            a = True        # why not true ?
            b = False       # why not false ?
            x = None        # none ?
        

So, in code which is in general lower case, those True/False/None looking really strange.

            def proc3(input_list):
                if len(input_list) == 0:
                    return None

                for i in range(0, len(input_list)):
                    for j in range(0, len(input_list)):
                        if input_list[i] == input_list[j] and i != j:
                            return False
                return True
        

OO but not OO

Being OO language, Python still relies on some procedural concepts. The good example is calculation of string length. I would expect, that string corresponding method len() or something, but instead we are having ‘global’ function that does that.

            s = "I'm string"
            print len(s)        # why not s.len() ?
        

len() is overloaded for other types, it would work for list and dictionaries as well,

            l = [1, 2, 3]
            d = { 1: 1, 2: 2, 3: 3 }
            
            print len(l)
            print len(d)
        

In the same manner, if you what to get reverse iterator for collection, I would assume there corresponding method that returns that iterator. Bad guess,

            l = [1, 2, 3]
            for e in reverse(l):    # why l.reverse()
                print e
        

__init__() method

Trying out classes in Python first you need to understand is how to construct an object. Python has constructor, that look like that,

            class MyClass:
                def __init__(self, a, b):
                    self.a = a
                    self.b = b
        

I could understand it would be init or _init or constructor, but I will probably never understand __init__ with 2 underscores before and after. It’s ugly.

Conclusions

I’m about to enroll to next Udactity courses, so my Python journey continues. I hope to get more in language and standard library.