Monthly Archives: September 2017

Best practices in software delivery process

Last Updated on by

Post summary: Short overview of a software delivery process which I consider very good and worth the “best practice” label that is being practised in a very successful software company.

Recently I finished an assignment in a company which I rate as the best I’ve worked so far in terms of software delivery process, individuals professionalism and company culture. Most of the things I’ve blogged about last 2.5 years I’ve heard, seen, learned and mastered working for that company. I decided to describe the process, because for me this is a very successful practice.

Background

Company provides B2B services by exposing a lot of APIs to its clients which then compose different functionality to their end customers. Business functionality is broken down into numerous amount of micro-services. Every micro-service is a separate project and is deployed on a separate machine. Those micro-services interconnect with each other and depend on each other. Micro-services are discovered through Netflix’s Eureka, no endpoint is ever hard-coded, except Eureka’s.

Technologies

There are different tools and frameworks used in order to deliver quality software on time. List of tools consist of following: Jira for project and issue tracking, Confluence for documents collaboration, Bamboo for continuous integration and deployment, Bitbucket (former Stash) for code reviews, HipChat or Slack for communication, SonarQube for static code analysis, Fortify for security static code analysis. Software code is stored in Git, written in Java, build with Gradle and deployed on Linux servers with Chef or Ansible.

Planning

In order to plan the work Agile methodologies are followed – Scrum or Kanban. There are external team of scrum masters which facilitate Scrum ceremonies and Scrum is being very dogmatic followed.

Development

Every story from the Jira board is developed in a separate branch. On every commit there is Bamboo plan that builds the branch, runs the unit tests and runs SonarQube static code analysis. In order to pass the build different code style rules should be met, also it is mandatory to have 80% code coverage of the unit tests. On each commit Jira number is put into commit comments. This provides traceability between Jira and tools like Bamboo and Bitbucket. Built artefacts are uploaded into Amazon S3 bucket where later they are used by Chef deployment. Each branch build can be deployed to Dev test node and tested by developer in a real environment. Branch can be merged to master only if there are two code reviews done by other team members. Code reviews are done with Bitbucket.

Testing

Main pillar of quality is the unit testing. Although JUnit is main framework some teams are using Spock and are very successful with it. Code coverage threshold is above 80%. Between 75% and 80% SonarQube reports warning, bellow 75% build fails and you cannot release. Some teams practice mutation testing with PITest to improve their unit testing. This definitely eliminates a lot of the bugs, but just unit testing is not enough. We have reached up to 97% code coverage (JUnit, Spock and PITest) by the unit testing and still have seen small bugs in production. Although there are no strict rules about it every team is required to have automated functional testing. It could be very basic, it could be very advanced, but in order to release functional tests should be green.

Deployment

Deployment is fully automatic using Chef. It is development team responsibility to prepare the cookbooks and provision test environments. Deployment is triggered by Bamboo deployment plan which calls Chef on the specified node. This makes the traceability between what Jira is being implemented, when it was build and when it was deployed, to which environment and in which build number.

Test environments

Apart from production there are three other test environments: Dev, QA and Staging. Each test environment can have one ore mode nodes. Each different micro-service provides at least one node in order to make complete and working B2B solution. Test nodes are in the cloud and their management is done with Scalr as well as custom framework that uses Amazon EC2 API and spins up nodes. Spinning up a new node is as simple as a button click. Before spinning a node test environment should be properly configured, this includes network, Chef cookbook, hardware capabilities, software setup, every details needed to have a ready to test environment. Each test environment have different purpose:

  • Dev – used by developers, main idea is to have some code committed into a branch, build it and deploy that branch to Dev environment in order to test with real dependencies given feature. Most micro-services have their test nodes working. Since there is a lot of development ongoing, sometimes happens that some micro-service is with incorrect version of is down.
  • QA – this is used mainly by QAs to verify build that is a candidate to go for a release. This environment is stable. All micro-services have test nodes and downtime is something exceptional. Data in this environment is dummy and incomplete one.
  • Staging – this is pre-production environment. It is mandatory each micro-service to have working node there. Data is in very mature state and more reliable than other environments.

Release process

Once feature is implemented, code reviewed and tested its branch can be merged to master. Once merged team can decide to release it to production right away or wait for more features to pile up and then release. In order to release there is separate Bamboo build plan that is run manually. It builds the master branch, runs SonarQube analysis, runs Fortify security scan, deploys to QA test environment and runs the functional tests. Then build is deployed to Staging and functional tests are run again. If everything is green at this point there is a stable release candidate. In order to release to production there is manual steps that has to be done. Release slot is negotiated with DevOps engineer. For every production deployment there should be DevOps standby if something goes wrong. Once DevOps time is provisioned then release request with proposed release time is made with information which is the Bamboo build plan that is released. This request is managed by separate team. They check what Jiras are being implemented, if all builds are green and if Staging deployment is green. If everything is green then release is approved. In release window deployment to production is made by team member with a single button click with Bamboo. In most of the cases everything is good, but in case of issues DevOps engineer has access to production nodes and can fix any issue. Important thing is that deployment is done firstly on one node, then this node is verified. In case there is issue with the new code, latest version can be reverted back to this node and release is aborted. If new code is OK then deployment can continue on other nodes with rate of 2-3-4 nodes at a time. The idea is not to have too many nodes down at a time.

Canary releases

Some features are way too big, way too risky or way to unpredictable how they will behave in production. In such cases there is practice of canary releases. Real production node is detached from load balancer and does not receive live traffic any more. New functionality is deployed there, it is evaluated by product owners, monitored by DevOps for issues. If functionality is OK then node can be attached to load balancer again and be left for some time to see how production traffic influences it.

Introducing a brand new micro-service

If new micro-service has to be introduced in then it should go through an architectural review. It is being evaluated what technologies are there how it operates and most important how it fits the micro-service landscape. There is team of architects that are responsible to keep landscape tidy and focused. There is extensive operational requirements checklist, such as: is HTTPS used, is logging following company standards, are passwords encrypted in DB, are sensitive configuration data encrypted on file system. There are many requirements that service should cover in order to go live. Even if it goes live first stage is a beta release where this service is exposed to selected number of partners which evaluate it first. Then it can be revealed to mass public.

Conclusion

I really enjoyed working for this company. It was a great learning opportunity because they keep up to date with new technologies and good practices. Processes and tools are constantly evolving keeping good quality of the code and the products. I definitely encourage to take a deep look, understand the process and eventually apply something into your software delivery process. Most important is the traceability that makes very transparent what feature is implemented, in which build deployed, etc. And traceability is something ISO auditors care very much about.

Read more...

Partial JSON deserialize by JsonPath with Json.NET

Last Updated on by

Post summary: Code examples how to deserialize only part of a big JSON file by JsonPath when using NewtonSoft Json.NET.

Code shown in examples bellow is available in GitHub DotNetSamples/JsonPathConverter repository.

Use case description

Imagine you have a big JSON which you want to deserialize into a C# object.

{
  "node1": {
    "node1node1": "node1node1value",
    "node1node2": [ "value1", "value2" ],
    "node1node3": {
      "node1node3node1": "node1node3node1value"
    }
  },
  "node2": true,
  "node3": {
    "node3node1": "node3SubNode1Value",
    "node3node2": {
      "node3node2node1": {
        "node3node2node1node1": [ 1, 2, 3 ]
      },
      "node3node2node2": "node3node2node1value"
    }
  },
  "node4": "{\"node4node1\": \"n4n1value\", \"node4node2\": \"n4n1value\"}"
}

File above is actually pretty small and used for demo purposes. In practice you can stumble upon terrifyingly big JSON files. NewtonSoft.Json or Json.NET is defacto the JSON standard for .NET, so it is being used to parse the JSON file. In order to deserialize this JSON to a C# object you need a model class that represent the JSON nodes. Although immense effort you can create such, but why bother if you are going to use just a fraction of all JSON data. This is where JsonPath comes in play. Json.NET allows you to query JSON by JsonPath, so one option is to manually query the JSON, find data you need and assign it to your C# object. This is not an elegant solution. Since query by JsonPath is possible this can be used in a JsonConverter that will automatically do the job. What is needed is a custom JsonPathConverter and a model class that will be deserialized to, both are described bellow.

JSON model class

It is easier to describe the JSON model first. Bellow is a code for JSON model class that will collect only data we need.

using System.Collections.Generic;
using Newtonsoft.Json;

namespace JsonPathConverter
{
	[JsonConverter(typeof(JsonPathConverter))]
	public class JsonModel
	{
		[JsonProperty("node1.node1node2")]
		public IList<string> Node1Array { get; set; }

		[JsonProperty("node2")]
		public bool Node2 { get; set; }

		[JsonProperty("node3.node3node2.node3node2node1.node3node2node1node1")]
		public IList<int> Node3Array { get; set; }

		[JsonConverter(typeof(JsonPathConverter))]
		[JsonProperty("node4")]
		public NestedJsonModel Node4 { get; set; }
	}

	public class NestedJsonModel
	{
		[JsonProperty("node4node2")]
		public string NestedNode2 { get; set; }
	}
}

JSON model class is annotated with [JsonConverter(typeof(JsonPathConverter))] which tells Json.NET to use JsonPathConverter class to do the conversion. JsonPathConverter is implemented in such a way that JsonProperty is a mandatory for each property in order to be parsed: [JsonProperty(“node1.node1node2”)].

JSON as a string

You may have noticed already the weird case where node4 in JSON file has actually a string value which is escaped JSON string. This is something unusual and may not be pretty good programming practice, but I’ve encountered it in a production code, so examples given here cover this weirdo as well. There is special NestedJsonModel class which this JSON string is being deserialized to.

JsonPathConverter

Code bellow implements JsonConverter abstract class and implements needed methods.

public class JsonPathConverter : JsonConverter
{
	public override bool CanWrite => false;

	public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
	{
		var jObject = JObject.Load(reader);
		var targetObj = Activator.CreateInstance(objectType);

		foreach (var prop in objectType.GetProperties().Where(p => p.CanRead && p.CanWrite))
		{
			var jsonPropertyAttr = prop.GetCustomAttributes(true).OfType<JsonPropertyAttribute>().FirstOrDefault();
			if (jsonPropertyAttr == null)
			{
				throw new JsonReaderException($"{nameof(JsonPropertyAttribute)} is mandatory when using {nameof(JsonPathConverter)}");
			}

			var jsonPath = jsonPropertyAttr.PropertyName;
			var token = jObject.SelectToken(jsonPath);

			if (token != null && token.Type != JTokenType.Null)
			{
				var jsonConverterAttr = prop.GetCustomAttributes(true).OfType<JsonConverterAttribute>().FirstOrDefault();
				object value;
				if (jsonConverterAttr == null)
				{
					serializer.Converters.Clear();
					value = token.ToObject(prop.PropertyType, serializer);
				}
				else
				{
					value = JsonConvert.DeserializeObject(token.ToString(), prop.PropertyType,
						(JsonConverter)Activator.CreateInstance(jsonConverterAttr.ConverterType));
				}
				prop.SetValue(targetObj, value, null);
			}
		}

		return targetObj;
	}

	public override bool CanConvert(Type objectType)
	{
		return true;
	}

	public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
	{
		throw new NotImplementedException();
	}
}

Deserialization work is done in public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) method. JSON is loaded to a NewtonSoft JObject and instance of result object is created. All properties of this result object are iterated in a foreach loop. It is important to not that properties should have both get and set in order to be considered in deserialization: objectType.GetProperties().Where(p => p.CanRead && p.CanWrite). If you have properties with just get or just set they will be ignored. JsonPropertyAttribute for each property is taken. If there is no such then exception is thrown. This part can be changed. JsonPath can be considered to be the property name: var jsonPath = jsonPropertyAttr == null ? prop.Name : jsonPropertyAttr.PropertyName. This is tricky though as C# is case sensitive and it might not work as property could start with capital letter, but JSON itself to be with lower case. Once there is JsonPath defined JObject is queried with jObject.SelectToken(jsonPath). This should return a valid token. In case of valid token result object property is checked for JsonConverterAttribute. If such exists then JSON is deserialized with this newly found JsonConverter instance. If there is no converter attached to this property then all existing converters are cleared and token is converted into object. Clearing part is important as in case of recursive call it will throw exception.

Usage

Once job above is done usage is pretty easy:

var fileContent = File.ReadAllText("jsonFile.json");
var result = JsonConvert.DeserializeObject<JsonModel>(fileContent);

result.Node1Array.Should().BeEquivalentTo(new List<string> {"value1", "value2"});
result.Node2.Should().Be(true);
result.Node3Array.Should().BeEquivalentTo(new List<int> { 1, 2, 3 });
result.Node4.NestedNode2.Should().Be("n4n1value");

Conclusion

In this post I have shown how to partially deserialize JSON by JsonPath picking only data that you need.

Read more...

Soft assertions for C# unit testing frameworks (MSTest, NUnit, xUnit.net)

Last Updated on by

Post summary: Code example of very easy and useful custom implementation of soft assertions in C# unit testing frameworks such as MSTest, NUnit or xUnit.net.

Code shown in examples bellow is available in GitHub DotNetSamples/SoftAssertions repository.

Unit vs Functional testing

Unit testing paradigm states that each test exercises particular code behaviour. So in a perfect world one unit test would have one assertion which defines unit test result – either passed or failed. This is why unit testing frameworks provide only asserts which stop further execution of current test method. In functional testing usually one test verifies several conditions. Not debating if this is good or bad. Assume you are doing GUI testing, once you have opened particular page you’d better do as much verification as possible to reduce the risk of bugs. Having this page opened over and over for each single check is not the most efficient way of testing. This is why when you run functional tests you need some kind of assert that indicates whether passed or failed but to let the test continue in no critical issue is present. Those are generally called “soft” asserts.

Soft assertions code

Following code is an implementations of soft assertions:

public class SoftAssertions
{
	private readonly List<SingleAssert> 
		_verifications = new List<SingleAssert>();

	public void Add(string message, string expected, string actual)
	{
		_verifications.Add(new SingleAssert(message, expected, actual));
	}

	public void Add(string message, bool expected, bool actual)
	{
		Add(message, expected.ToString(), actual.ToString());
	}

	public void Add(string message, int expected, int actual)
	{
		Add(message, expected.ToString(), actual.ToString());
	}

	public void AddTrue(string message, bool actual)
	{
		_verifications
			.Add(new SingleAssert(message, true.ToString(), actual.ToString()));
	}

	public void AssertAll()
	{
		var failed = _verifications.Where(v => v.Failed).ToList();
		failed.Should().BeEmpty();
	}

	private class SingleAssert
	{
		private readonly string _message;
		private readonly string _expected;
		private readonly string _actual;

		public bool Failed => _expected != _actual;

		public SingleAssert(string message, string expected, string actual)
		{
			_message = message;
			_expected = expected;
			_actual = actual;
		}

		public override string ToString()
		{
			return $"'{_message}' assert was expected to be '{_expected}' " +
				$"but was '{_actual}'";
		}
	}
}

Soft assertions details

Actual assertion is handled by SingleAssert class. It contains a message to be displayed to user in case of fail as well as expected and actual values. They are stored as strings. All asserts during testing are stored in a List<SingleAssert>. There are several methods that add assert. There are such that accept bool, string and int. You can extend and add as many as you want. It is mandatory to call AssertAll() method so asserts can be evaluated. Evaluation consists of filtering out passed asserts leaving only failed: var failed = _verifications.Where(v => v.Failed).ToList(). Then list with failed is checked for empty failed.Should().BeEmpty(). In this case FluentAssertions framework is used, but code can be changed to such that suits your particular needs.

Soft assertions usage

Usage is pretty straight forward. SoftAssertions object should be created before each test and asserted after each test:

[TestClass]
public class UnitTest
{
	private SoftAssertions _softAssertions;

	[TestInitialize]
	public void SetUp()
	{
		_softAssertions = new SoftAssertions();
	}

	[TestCleanup]
	public void TearDown()
	{
		_softAssertions.AssertAll();
	}

	[TestMethod]
	public void TestMixedSoftAssertions()
	{
		_softAssertions.Add("Passing bool Add assertion", true, true);
		_softAssertions.Add("Failing bool Add assertion", true, false);
		_softAssertions
			.Add("Passing string Add assertion", "SameString", "SameString");
		_softAssertions
			.Add("Failing string Add assertion", "SameString", "OtherString");
		_softAssertions.Add("Passing int Add assertion", 1, 1);
		_softAssertions.Add("Failing int Add assertion", 1, 2);
		_softAssertions.AddTrue("Passing AddTrue assertion", true);
		_softAssertions.AddTrue("Failing AddTrue assertion", false);
	}
}

Soft assertions result

Result of test shown above is: Result Message: Expected collection to be empty, but found {‘Failing bool Add assertion’ assert was expected to be ‘True’ but was ‘False’, ‘Failing string Add assertion’ assert was expected to be ‘SameString’ but was ‘DifferentString’, ‘Failing int Add assertion’ assert was expected to be ‘1’ but was ‘2’, ‘Failing AddTrue assertion’ assert was expected to be ‘True’ but was ‘False’}.

This comes out of the box because FluentAssertions is used. Otherwise you have to do some other output and assertions.

Other soft assertions

Some custom implementation of soft assertions is as well available in NTestRunner framework, but it is more complex and demanding special approach for writing tests.

Conclusion

Soft assertions are very useful in functional testing. With this simple class you can directly have them in your functional tests.

Read more...