Integrating TFS with Jenkins

Hi Everyone,

As you all know I'm no great fan of TFS. Since I last talked about TFS things have improved somewhat.

Let me tell you a brief story.

PART 1

Two weeks ago I was doing a very quick engagement for a utility company, mostly to help out a friend more than anything. The engagement was only for three days. As they had no CI capability I set up a Jenkins box and tried to integrate with the Bonobo Git server on the same box. Jenkins failed check it out the code...we think because certificates needed to be set up. Due to time constraints we got approval to move the code to a private GitHub repo. Problem solved.

PART 2

Just this week, at my latest engagement (in the same building I'd just left 3 weeks earlier but that's another story!), I was confronted with a TFS server and a request to set up a CI build. I recommended Jenkins and Git. I expected they would take more convincing to move away from TFS as a build server, however fortunately they did their own research and came around to my thinking that TFS as a build server is just not as good as Jenkins. The difficult part was that private GitHub repositories would not be allowed in this corporate environment. (This corporate is very behind the times!)

PART 3

We struggled for a couple of days to setup a git repo we could control in the (overly locked down) dev environment that we had been given. In the end we abandoned this endeavor...we tried all manner of things, even a git backed repository in TFS 2013...just too hard.

I thought I'd take another look at the TFS plugin for Jenkins and was very pleased to see that it has recently been through a major rewrite. Version 4.0 of the plugin was released on August 27th, 2015....and it works!

The trick is to get the settings "just so".

1. In the job config select the Team Foundation Server option from the Source Management section.
2. In the Server URL field enter https://<tfs-server-name>:<port>/tfs/<collection-name>
3. In the Project path field enter $/path/to/root/of/my/solution
4. In the User name field enter the user name that has access to TFS
5. In the User password field enter the password associated with the user name

I didn't enter anything else.

Points to note:

• The Server URL has to be html escaped. So, for instance my <collection-name> had spaces in it, so My Collection Name becomes My%20Collection%20Name. If your <tfs-server-name> or <collection-name> has other special characters they will also have to be correctly escaped.
• The User name doesn't have to be an email address. In my case it wasn't.

If you don't get the above settings correct (I initially didn't know I had to include the collection name), you'll probably get an error such that it can't find a SOAP endpoint.

Get it correct, however and BAM! your Jenkins workspace will be populated with your source code.

Now it is easy integrate Jenkins with TFS such that you can get the code out of TFS and work with it in a Jenkins build.

Hopefully this has been helpful to someone out there. Let me know in the comments. I always like to hear your feedback.

Till next time...

Selenium Testing with SauceLabs, NUnit and Saucery

Welcome to the next installment of JenkinsHeaven,

I've written previously on how to install the Jenkins Selenium plugin to turn your Jenkins into a SeleniumGrid installation.

This is certainly how we used to do it a few years ago.  That approach requires you to maintain VM infrastructure to have machines running the various versions and type of browsers.  With so many target platforms available and required these days this gets unmanageable quickly.  Not practical.

The solution to achieving cross-browser AND cross-platform testing is to go convince your manager to invest in a SauceLabs testing account.

SauceLabs provides a SeleniumGrid in the cloud and has literally hundreds of platforms available in pristine (on a per test basis) VMs.  With a SauceLabs account you can test your application across all these platforms.

Go do that now and I'll wait...

...ok, you're back? I trust you successfully convinced them of the considerable value to be added.

With a Saucelabs account, cross-browser testing is achievable.

Now with all these available platforms, your next step will probably be to agree with your manager what the corporate-defined list of supported platforms will be.  What platforms will you need to test against? What platforms will the organisation support?  These questions will be answered jointly by the IT department and key stakeholders from the business.

Why don't you do that now and I'll go get a coffee...

...Ok, what did you come up with?

The team at Full Circle Solutions has been through this process on a few occasions already and we know you can easily come up with 50-75 platforms without breaking a sweat by the time you include a few popular desktop browsers and some Apple and Android mobile platforms. As with NUnit Parameterised tests, the combinations blow out quickly.

With this shiny new mandate in hand and the blessing of your manager. all you have to do is request this set of desired platforms as part of your test [Setup] method before each test.

How to do this?

You could google it...

You could even trawl through SauceLabs documentation...

What you will quickly realise however is that targeting your newly agreed on set of platforms in a holistic, non-piecemeal manner isn't easy.

Saucery For NUnit knows all the rules to request platforms correctly so you don't have to.  Saucery For NUnit allows your team to focus on writing your test cases so you can deliver code coverage to the business faster.

The How To page explains the detail of integrating the NuGet package into your test suites.  Make sure you upgrade to the latest version and purchase an activation key.

Having just invested a couple of thousand dollars to get a SauceLabs account, only $99 invested in Saucery for NUnit to unlock SauceLabs and solve the DesiredCapability problem is well worth it and will pay you back in spades.

Till next time...

Automated Testing with Jenkins and C#: StackedBrowsers Revisited

Welcome to the next installment of JenkinsHeaven.

You might be wondering what Merlin has got to do with automated testing.  Good question, we'll get to that.

Today is a special post. Since the last post Full Circle Solutions has been hard at work on Project Merlin, hence the hiatus from writing.

Project Merlin is the complete rewrite of StackedBrowsers (to be more exact the SauceLabs half of StackedBrowsers).

It has been architected to operate within the context of Jenkins and the Sauce OnDemand plugin.

Full Circle Solutions will be launching Saucery soon.

UPDATE 20th November 2014 - Saucery For NUnit has been released and is available on NuGet. Get your activation key here.

Were currently hard at work on Saucery for JUnit (coming soon).

We would love to hear your feedback and questions either in the comments section here or on the contact form on our website.

Take a look at Full Circle Solutions for more.

Till next time...

Automating .NET builds – It’s not all about TFS

There is another way. You have an option. Just because you are working in a .NET shop does not mean you are locked into using TFS.

In fact in many ways you would be better to break the shackles of TFS. It’s easy to use Jenkins for Continuous Integration and Git for configuration management with.NET.

This essay will not try to detail all the reasons why TFS is bad. Google “TFS sucks” and you’ll get “About 200,000 results”. Pages like this, this, and, I can’t resist, this do that job nicely.

Rather, this piece will discuss:

• Prevailing attitudes to looking outside the box.
• The current industry adoption of Jenkins and (relatedly) Git.
• Why Microsoft has (again) lost the war and is capitulating to the Open Source community.

While in the open source community there is a constant curiosity to try out new things and apply technologies in ways they weren’t initially intended, the prevailing attitude to trying something that does not have the Microsoft holy seal of approval seems to be “But we’re a Microsoft shop”. Hands up if you’ve experienced the same response.

It is probably unsurprising then that the penetration of Jenkins into the .NET community is shallow. I am encouraged however, as the author of the JenkinsHeaven blog that over 4000 people a month are interested in using Jenkins to do .NET builds and that the current situation may be changing. Go see those posts for the how-to of Jenkins and .NET.

Microsoft capitulates (again)

Rather than "All things to all people", it is probably more apt to say of TFS: “You can’t fool all the people, all the time”. In short, TFS tries to do too much and as a result is not best-of-breed at anything. I give no credence to the argument that TFS is an Application Lifecycle Manager. If anything this only proves my point. At a high level, Microsoft seems to have dropped the Separation of Concerns or Single Responsibility Principle (take your pick) ball when it comes to TFS.

Cracks (or is it hypocrisy?) started to appear about a year or so ago when CodePlex migrated all their repositories to Git….”Say what?” you say. Yes indeed! CodePlex migrated all their repositories to Git. In other words, CodePlex is an interface to Git.

Why should I adopt TFS when Microsoft has quietly adopted Git? Furthermore, Visual Studio 2013 has dropped the pretence and is adopting Git in a big way. All the details are here. Mendocino Triple Junction is closer to Seattle than we thought. It is with this development that Microsoft has announced its capitulation. Git has won the war and Git will rule the world.

After trying for a couple of weeks at work to integrate Jenkins into the build environment of a .NET project the piece in the puzzle that I couldn’t solve was TFS. I raised this (mainly to see if anyone else had experienced this and had a workaround). As far as I can gather the issue is with TFS, not Jenkins or the plugin. TFS does not want to play with other children.

That is why Microsoft adopting Git is a wonderful thing. It will hopefully take the TFS Source control barrier out of the Jenkins-as-CI-for-.NET-environment equation. In other words, it is my hope that this development will drive Jenkins adoption in .NET shops.

In the same way that Nirvana gave rock and roll its requisite periodic kick in the nuts, all I would say is that it is perhaps disappointing that Microsoft always seems to trail the field rather than leading the way, with a modus operandi of cherry picking 5 to 10 year old open source ideas.

Thank you for reading this far, if I have encouraged you to question the TFS status-quo in your .NET projects, then I have achieved my aim.

You may have got the impression reading this that I am a Microsoft hater. Far from it! As the Founder and Lead Developer of Full Circle Solutions, Visual Studio is our tool of choice. When it comes to writing code that is. It is worth noting at this point that before being a .NET developer, I was a Java developer. C#’s other name being “I-can’t-believe-its-not-Java”. As I see it .NET sits in the same space as J2EE and .NET is better documented and much more polished.

Visual Studio 2012 and the .NET framework are the start and end of our Microsoft tool adoption. We use Jenkins, not TFS, for Continuous Integration to ensure the quality of our product offerings. We use Git, not TFS, for configuration management.

It is due to my experience in both the Open Source and .NET world and thinking about this area while researching and writing JenkinsHeaven that, 3 years ago, I settled on Jenkins, Git and Visual Studio as the best of breed toolset. This is the direction I have selected for Full Circle Solutions.

Till next time...

Celebrating 100,000

Hi Everyone,

I'd like to take a moment to note that today JenkinsHeaven had its 100,000th page view.

Who would have thought it! I certainly wasn't expecting it. I am more than a little amazed.

I have endeavored to post content that is not only fun to read but is also useful and quality material. I have taken a hiatus from writing posts as I am focused on finding a new job since being made redundant from my last employer after 12 years. I will continue upholding my own high standards with quality posts you have come to expect once I find my next role.

Thank you everyone for your continued support.

Till next time.

Selenium testing in the cloud with BrowserStack

Hello fellow Jenkins fans, I'm back for another instalment of JenkinsHeaven.

This post is a little off topic but is so seriously cool it has to be covered.

In the past, to perform cross-browser, or better yet cross-platform testing, you would install the Selenium Grid plugin into your Jenkins master and have a number of physical (or virtual) machines with different browsers installed on them and set up some tags so that the Selenium Grid could find them. Once all this infrastructure was set up you would write your Selenium tests.

Side Note: We're all doing cross-platform testing as part of our builds right? If you answered "Wrong!" like Arnie in 1985's Commando. I highly recommend that you integrate cross-platform testing in all your web application builds before the 11 hours is up (if you're confused, watch the movie).

While the traditional approach of in-house testing slave infrastructure is still valid and might be necessary for your organisation, with Browserstack you can put all the infrastructure in the cloud. They take care of it for you. Browserstack makes it really easy. They essentially provide the functionality of Selenium Grid, the test slaves and tags.

There are two main ways of interacting with browserstack: Automate and ScreenshotsAPI. There is "Live" testing but in my opinion its only really useful for concept demonstrations. Its not programmatic like the first two and we're interested in the programmatic stuff. Responsive appears to me to be the same as ScreenshotsAPI.

They support a ton of browsers and platforms and you can even test non-public facing urls (like test environments) with the web tunnel functionality. DANGER WILL ROBINSON: Be careful to note that the browsers and platforms supported by ScreenshotsAPI is not the same as Automate. They are slightly different (at least that is the case at the time of writing).

Happily, all this leaves you to focus on writing your tests. With only a few tests written, the effect of configuring just some of the supported platforms means that in a short amount of time you find yourself with a test suite with hundreds of combinations. Tests X Browsers X Operating Systems. :D

I found that while they give you automate code examples (Python, Ruby, Java, C#, PHP, Perl, Node.js) which are a helpful bootstrap it takes a bit of work to understand exactly what you can and can't do. For instance, I tried for 2 days get a batch file to call the third-party ScreenShooter tool repeatedly. Even with the /w argument it won't work after the first iteration (at least I haven't been able to yet.). BrowserStack's support team advised me that their servers limit one screenshot call at a time. You won't find that in their documentation.

While the documentation on the BrowserStack website isn't too bad I feel it is lacking in some areas.

I wrote stacked-browsers to pull all the threads together and make your lives easier...that...and my current project needed it.

stacked-browsers implements different flavors of integration with BrowserStack: Java, CSharp and Ruby (via ScreenShooter. I am yet to get the Screenshots API Ruby Library to work). If you would like to implement testers for the other languages feel free to send me a pull request. The structured readme at each level of the source tree should explain everything from setup to building to running. Take note of the batch files I wrote to make these steps easy.

It is cool to execute the TestRunner from the command line and see the test session appear almost immediately on Browserstack.com on the Automate page with the name you specified.

Browserstack's example code has 16 degrees of parallelism. I noticed in the top corner of the account I was using that they afforded me 5 degrees of parallelism. I had mixed results with this. With 2, I only seemed to have issues with one of the one of the Samsung android platforms, with 5 set, I had 8 tests running at times and had more timeout issues. I also noticed after about 10 minutes of my session running that this dropped to 2 test running anyway. I concluded that BrowserStack was throttling my session. I asked about this but their support team are yet to confirm or deny.

For a quick win, I chose Java Parallel JUnit (scroll down to "JUnit test for running in parallel browsers" for the helper class) as the initial mode of choice, but I expect the other flavors would work equally easily. Overall, I am very impressed with how smoothly BrowserStack works and would recommend it to anyone wanting to do cross-platform testing.

DISCLAIMER: I am neither employed by BrowserStack nor being paid for these comments (I am not a Sydney radio host...you know what I mean). I just think it works really well.

Oh, and if anyone manages to write a batch file that successfully calls the ScreenShooter tool or implement a client for the Screenshots API Ruby Library (or any other improvements) send me a pull request for the stacked-browsers repo.

I hope you have found this useful.

UPDATE: Saucery2 For NUnit is the commercial version of Stacked Browsers.

Till next time...

Introducing Full Circle Solutions - solutions from every angle

Today it is my privilege to announce the opening of Full Circle Solutions.

Our mission is to conceive of and provide innovative applications that make the lives of businesses and developers alike easier.

We use Jenkins internally along with extensive automated tests (we believe in dog fooding!!) to ensure the quality of our product offerings.

Visit Full Circle Solutions and peruse our offerings. Vote on upcoming features, and make suggestions for features you would like to see added to our "to do" list.

We will continue to blog about Jenkins and its industry adoption while we continue working on innovative products to make life easier.

Till next time...