Is TDD skill actually required by employers?
Is TDD popular among developers? Do managers knows about benefits of TDD? Are employers really looking for TDD skilled people?
I was thinking about such kind of questions and decided to perform my initial research. My research was quite simple, I wanted to check popular job looking sites and review latest job offers, especially for “skills” sections. How many of employers actually seek for developers who know/use/love TDD. Since I’m geek I would not do it manually, so I’ve written an application for that. Crawler that could get data from job looking sites and store it to DB for further analisys. I already got the data and would like to share it in this post.
How it works?
Like any other crawler it has one big cycle that makes a web requests to site,
gets response, extracts the links and data from response, stores data somewhere. It proceeds as soon as relevant data is present on pages. Vacancy crawler does a search request, extracts links for all vacancy pages. As soon as link extracted, it does request to vacancy page. It analyzes the body of vacancy description by very simple method: searching for a keywords it text. So, to detect is TDD skill required or not, crawler try to match some words from vocabulary.
Similar approach used to understand what technology skills (.NET, Java, C++) is required. At final it creates a record contains site name, position, technology used and TDD demand flag and stores data to database.
Source of information
I’ve taken two sites as source of information. First, RabotaUa (Ukrainian one, Ukraine is
one of big players of IT outsourcing in Europe, so data will be really relevant). Second one, I wanted to pick up from USA, but it was difficult to find it, since I’m not aware of its popularity, reputation and so on. I even asked question of StackOverflow, but my question was closed. I choose JobsForProgrammers as one of google suggested.
RabotaUa results
I’ve extracted 978 records from RabotaUa. It is latest, actual, recently posted job offers. 150 of 987 vacancies contained requirements for TDD skills.
How many of TDD skills required per technology?
PrgJobs results
I’ve extracted 1000 records from JobsForProgrammers. Crawler could proceed more, but I’ve noticed that on latest pages, site contains not really relevant data, not developers jobs and offers with short and not always adequate description. So, I still consider to crawle some other USA site for data. Anyway, here are results. Only 69 of 1000 requires TDD, 7%!
Technologies breakdown, also a bit strange. Match for technology were difficult, for several reasons. Job description headline, usually contained to generic description (as Software Developer, Web developer and so on), job description body contained multiskills requirements (like C++/Perl, or C#/Java, VB.NET/Java) that current version of crawler could not handle properly.
Conclusions
To be honest, I would not expect such data. I thought 40-60% should ask for TDD, but we see that it is less than 16% from
Ukranian data source and less than 7% for USA. For me, as TDD follower this is really disappointing results. I realize that such results are very simple and rough, could not be used for some real life analytics, but it gives
a vision, for sure.
Also I plan to do several technical blogs with details of implementation of Crawler, I’ve created for this report.
Please let me know, what you think about such results, what further improvements could be done for more fine results, what other data sources could be used?
Update
Is TDD popular among developers? Do managers knows about benefits of TDD? Are employers really looking for TDD skilled people?
I was thinking about such kind of questions and decided to perform my initial research. My research was quite simple, I wanted to check popular job looking sites and review latest job offers, especially for “skills” sections. How many of employers actually seek for developers who know/use/love TDD. Since I’m geek I would not do it manually, so I’ve written an application for that. Crawler that could get data from job looking sites and store it to DB for further analisys. I already got the data and would like to share it in this post.
How it works?
Like any other crawler it has one big cycle that makes a web requests to site, gets response, extracts the links and data from response, stores data somewhere. It proceeds as soon as relevant data is present on pages. Vacancy crawler does a search request, extracts links for all vacancy pages. As soon as link extracted, it does request to vacancy page. It analyzes the body of vacancy description by very simple method: searching for a keywords it text. So, to detect is TDD skill required or not, crawler try to match some words from vocabulary. Similar approach used to understand what technology skills (.NET, Java, C++) is required. At final it creates a record contains site name, position, technology used and TDD demand flag and stores data to database.
Source of information
I’ve taken two sites as source of information. First, RabotaUa (Ukrainian one, Ukraine is one of big players of IT outsourcing in Europe, so data will be really relevant). Second one, I wanted to pick up from USA, but it was difficult to find it, since I’m not aware of its popularity, reputation and so on. I even asked question of StackOverflow, but my question was closed. I choose JobsForProgrammers as one of google suggested.
RabotaUa results
I’ve extracted 978 records from RabotaUa. It is latest, actual, recently posted job offers. 150 of 987 vacancies contained requirements for TDD skills.
How many of TDD skills required per technology?
PrgJobs results
I’ve extracted 1000 records from JobsForProgrammers. Crawler could proceed more, but I’ve noticed that on latest pages, site contains not really relevant data, not developers jobs and offers with short and not always adequate description. So, I still consider to crawle some other USA site for data. Anyway, here are results. Only 69 of 1000 requires TDD, 7%!
Technologies breakdown, also a bit strange. Match for technology were difficult, for several reasons. Job description headline, usually contained to generic description (as Software Developer, Web developer and so on), job description body contained multiskills requirements (like C++/Perl, or C#/Java, VB.NET/Java) that current version of crawler could not handle properly.
Conclusions
To be honest, I would not expect such data. I thought 40-60% should ask for TDD, but we see that it is less than 16% from Ukranian data source and less than 7% for USA. For me, as TDD follower this is really disappointing results. I realize that such results are very simple and rough, could not be used for some real life analytics, but it gives a vision, for sure.
Also I plan to do several technical blogs with details of implementation of Crawler, I’ve created for this report.
Please let me know, what you think about such results, what further improvements could be done for more fine results, what other data sources could be used?
Update