Advanced WPF automation – memory usage

Last Updated on by

Post summary: Highlight eventual memory issue when using Telerik Testing Framework and TestStack White for desktop automation.

Memory is an important aspect. When you have several test cases it is not a problem. But on large projects with many tests memory turn out to be a serious issue.

Reference

This post is part of Advanced WPF desktop automation with Telerik Testing Framework and TestStack White series. The sample application can be found on GitHub.

Problem

Like every demo on certain technology automating WPF applications looks cool. And also like every technology problems occur when you start to use it on a large scale. Problem with WFP automation with Telerik Testing Framework and TestStack White is the memory. When your tests’ number grows frameworks start to use too much memory. By too much, I mean over 1GB which might not seem a lot but for a single process actually is. Increasing RAM of test machine is an only temporary solution and is not one that can be scaled.

Why so much memory

I have a project with 580 tests and 7300 verification points spread in 50 test classes. I’ve spent lots of hours debugging and profiling with several .NET profiling tools. In the end, all profilers show that a large amount of memory used is in unmanaged objects. So generally there is nothing you can do. It seems like some of the frameworks or both have memory issues and do not free all memory they use.

Solution

The solution is pretty simple to suggest but harder to implement – run each test class in the separate process. I’m using a much more enhanced version of NTestsRunner. It is running each test class in separate windows process. Once the test is finished results are serialized in results directory, the process is exited and all memory and object used for this test are released.

Conclusion

The memory could be a crucial factor in an automation project. Be prepared to have a solution for it. At this point, I’m not planning to put running tests in a separate process in NTestsRunner. If there is demand it is a pretty easy task to do it.

Related Posts

Read more...

Complete guide to email verifications with Automation SMTP Server

Last Updated on by

Post summary: How to do complete email verification with your own Automation SMTP server.

SMTP is protocol initially defined in 1982 and is still used nowadays. In order to automate application which sends out emails, you need SMTP server which reads messages and saves them to disk for further processing. Note that this is only in the case when your application sends emails.

Windows SMTP server

One option is to use SMTP server provided by Windows. Problems here are two. First is that from Vista SMTP server is no more supported. There is SMTP server in Windows Server distributions but the license for them is more expensive. The second problem comes from the configuration of the server. You might have several machines and configurations should be maintained on all of them. It is a feasible option to use Windows SMTP server but the current post is not dedicated to it.

Automation SMTP Server

What I offer in this post is your own Automation SMTP Server. It is located in following GitHub project. The solution is actually a mixture of two open source projects. For the server, I use Antix SMTP Server For Developers, which is really good SMTP server. It is windows application and is more suitable for manual SMTP testing rather than automation. I’ve extracted the SMTP core with some modifications as a console application which saves emails as EML file on disk. For the reading of emails, I use the source code of Easily Retrieve Email Information from .EML Files article with several modifications. What you need to do in order to make successful email verification is download executable from GitHub and follow instructions below. More info for it can be found on its homepage Automation SMTP Server.

Automation SMTP Server usage

In GitHub AutomationSMTPServer repository there is an example that shows how to use Automation SMTP Server. The server should be added as a reference to your automation project. Since it is a reference it gets copied into compiled executables folder.

Delete recent emails

Before doing anything in your tests it is good to delete old emails. Automation SMTP Server is saving mail into a folder named “temp”. This is how it works and cannot be changed.

private string currentDir =
	Directory.GetCurrentDirectory() + Path.DirectorySeparatorChar;
private string mailsDir = currentDir + "temp";

if (Directory.Exists(mailsDir))
{
	Directory.Delete(mailsDir, true);
}

Start Automation SMTP Server

The server is a console application. It receives emails and saves them to disk. If counterparty sends a QUIT message to disconnect server gets restarted to wait for next connection. The server should be started as a process. Port should be provided as arguments. If not provided it can be configured in SMTP Server config file. If not configured there it gives a message and takes 25 for default port.

Process smtpServer = new Process();
smtpServer.StartInfo.FileName = currentDir + "AutomationSMTPServer.exe";
smtpServer.StartInfo.Arguments = "25";
smtpServer.Start();

Send emails

This is the point where your application under test is sending emails which you will later verify.

Read emails

Once emails have been sent out from the application under test you are ready to read and process them.

string[] files = Directory.GetFiles(mailsDir);
List<EMLFile> mails = new List<EMLFile>();

foreach (string file in files)
{
	EMLFile mail = new EMLFile(file);
	mails.Add(mail);
	File.Delete(file);
}

Verify emails

Here you can use EMLFile class which is parsing the EML file and is representing is an object so you can do operations on it. Once you have the mail as an object you can access all its attributes and verify some of them. It all depends on your testing strategy. Another option is to define on expected EML file, read it and compare both actual and expected. EMLFile class has predefined Equals method which is comparing all the attributes of the emails.

bool compare1 = mails[0].Equals(mails[1]);
bool compare2 = mails[0].Equals(mails[2]);
bool compare3 = mails[1].Equals(mails[2]);

Stop Automation SMTP Server

This part is important. If not stopped server will continue to work and will block the port. Its architecture is defined in such manner that only way to stop it is it to terminate console application. In a case where you have started it from C# code as process way to stop it is to kill the process.

smtpServer.Kill();

Conclusion

Proper email verification can be a challenge. In case your application under tests send emails I would say it is crucial to have correct email testing as mail is what customers receive. And in the end, it is all about customers! So give it a try and enjoy this easy way of email verification.

Read more...

Extract and verify text from PDF with C#

Last Updated on by

Post summary: How to extract text from PDF in C#.

PDF verification is pretty rare case in automation testing. Still it could happen.

iTextSharp

iTextSharp is a library that allows you to manipulate PDF files. We need very small of this library. It has build in reader that iterates through pages and returns only text.

using iTextSharp.text.pdf;
using iTextSharp.text.pdf.parser;
using System.Text;

namespace PDFExtractor
{
	public class PDFExtractor
	{
		public static string ExtractTextFromPDF(string pdfFileName)
		{
			StringBuilder result = new StringBuilder();
			// Create a reader for the given PDF file
			using (PdfReader reader = new PdfReader(pdfFileName))
			{
				// Read pages
				for (int page = 1; page <= reader.NumberOfPages; page++)
				{
					SimpleTextExtractionStrategy strategy =
						new SimpleTextExtractionStrategy();
					string pageText =
						PdfTextExtractor.GetTextFromPage(reader, page, strategy);
					result.Append(pageText);
				}
			}
			return result.ToString();
		}
	}
}

Verification

Once extracted text can be verified against expected as described in Text verification post.

Related Posts

Read more...

Text verification

Last Updated on by

Post summary: Verify actual text with expected one by ignoring what is not relevant during compare.

In automation testing, there is no definitive way what text verification is best to be done. One strategy is to check that an expected word or a phrase exists in actual text shown in the application under test. Another strategy is to prepare a large amount of text to verify. Later strategy is expensive in case of effort for preparation and maintenance. The first strategy might not be sufficient to do correct verifications.

In between

What I suggest here is something in between. Not too much but not too less. Problem with a paragraph of text to be verified is it might contain data we do not have control over, e.g. date, time, unique values, etc.

Example

Imagine an e-commerce website. When you place the order there is order confirmation page. You want to verify not only that you are on this page but also that text is correct as per specification. Most likely text will contain data you do not have control over – order number and date. Breaking verification is small chunks is an option. Another option is to manipulate the actual text. The third option is to define the text as expected with special strings that will get ignored during compare.

Actual vs Expected

Actual text could be: “Order 123456 has been successfully placed on 01.01.1970! Thank you for your order. ”
The expected text could be: “Order ~SKIP~ has been successfully placed on ~SKIP~! Thank you for your order. ”
And then you can compare both where ~SKIP~ will be ignored during compare.

Compare code

Code to do the compare shown above is incorporated in NTestsRunner also:

public const string IgnoreDuringCompare = "~SKIP~";

public static bool EqualsWithIgnore(this string value1, string value2)
{
	string regexPattern = "(.*?)";
	// If value is null set it to empty
	value1 = value1 ?? string.Empty;
	value2 = value2 ?? string.Empty;
	string input = string.Empty;
	string pattern = string.Empty;
	// Unify new lines symbols
	value1 = value1.Replace("\r\n", "\n");
	value2 = value2.Replace("\r\n", "\n");
	// If no one conains ignore string then compare directly
	if (!value1.Contains(IgnoreDuringCompare) &&
		!value2.Contains(IgnoreDuringCompare))
	{
		return value1.Equals(value2);
	}
	else if (value1.Contains(IgnoreDuringCompare))
	{
		pattern = Regex.Escape(value1).Replace(IgnoreDuringCompare, regexPattern);
		input = value2;
	}
	else if (value2.Contains(IgnoreDuringCompare))
	{
		pattern = Regex.Escape(value2).Replace(IgnoreDuringCompare, regexPattern);
		input = value1;
	}

	Match match = Regex.Match(input, pattern);
	return match.Success;
}

Use in tests

In your tests you will do something like:

string actual = OrderConfirmationPage.GetConfirmationText();
string expected = "Order " + ExtensionMethods.IgnoreDuringCompare +
	" has been successfully placed on " + ExtensionMethods.IgnoreDuringCompare +
	"! Thank you for your order. ";
Assert.IsTrue(actual.EqualsWithIgnore(expected));

Conclusion

It might take little bit more effort to prepare expected strings but verification will be more accurate and correct rather than just to expect a word or a phrase.

Related Posts

Read more...

Advanced WPF automation – read dependency property

Last Updated on by

Post summary: What is dependency property in .NET and how to read it from Telerik Testing Framework.

In this post, I’ll show an advanced way of getting more details from an object and sophisticate your automation.

Reference

This post is part of Advanced WPF desktop automation with Telerik Testing Framework and TestStack White series. The sample application can be found in GitHub SampleAppPlus repository.

Dependency property

Dependency properties are an easy way to extend available in .NET framework functionality. In SampleAppPlus there is CustomControl defined. Purpose of this control is to store text and visualize this text as image. The text is stored in dependency property.

public partial class CustomControl : UserControl
{
	public static readonly DependencyProperty MessageProperty =
			DependencyProperty.Register("Message",
									typeof(string), typeof(CustomControl),
									new PropertyMetadata(OnChange));

	...

	public string Message
	{
		get { return (string)GetValue(MessageProperty); }
		set { SetValue(MessageProperty, value); }
	}

	...

}

Read dependency property

In order to be able to properly automate something, you have to know the internal structure of the application. Generally, you will try to locate and read the element and it will not work in the ways you are used working with elements. At this point, you have to inspect the source code of application under test and see how it is done internally. Most important if dependency property is used you should know its name. Once you know the name reading is easy.

public class MainWindow : XamlElementContainer
{
	...

	private UserControl CustomControl_Image
	{
		get
		{
			return Get<UserControl>(mainPath + "CustomControl[0]");
		}
	}

	public Verification VerifyCustomImageText(string expected)
	{
		string actual =
			CustomControl_Image.GetAttachedProperty<string>("", "Message");
		return BaseTest.VerifyText(expected, actual);
	}
}

GetAttachedProperty

GetAttachedProperty is a powerful method. Along with reading dependency properties, you can read much more. In some cases, WPF elements are nested in each other or in tooltip windows. In other cases, some object is bound to WPF element. In such situations you can try to access the elements and method will return you FrameworkElement object. From this object, you can again get GetAttachedProperty to access some class specific property. In all cases, you will need access to the application under test code to see how it is working internally.

FrameworkElement tooltip = wpfElement.
	GetAttachedProperty<FrameworkElement>("", "ToolTip");
string value = tooltip.GetAttachedProperty<string>("", "SomeSpecificProperty");

Conclusion

GetAttachedProperty is a powerful method. Once you get stuck with normal processing of elements you can always try it. I would say definitely give it a try.

Related Posts

Read more...

Advanced WPF automation – working with WinForms grid

Last Updated on by

Post summary: Example how to work with WinForms grid with TestStack White.

TestStack White is a really powerful framework. It works on top of Windows UI Automation framework hiding its complexity. If White is not able to locate element you have access to underlying UI Automation and you can do almost anything you need.

Reference

This post is part of Advanced WPF desktop automation with Telerik Testing Framework and TestStack White series. The sample application can be found on GitHub.

MainGrid

For single responsibility separation grid logic is in separate class MainGrid.cs. The constructor takes White.Core.UIItems.WindowItems.Window object. Inside the window, we search for an element with control type ControlType.Table. It is the only one of its kind. If there are more we should narrow down the SearchCriteria.

public class MainGrid
{
	private Table table;
	public MainGrid(Window window)
	{
		SearchCriteria search = SearchCriteria.ByControlType(ControlType.Table);
		table = window.Get<Table>(search);
	}

	public string GetCellText(int index)
	{
		TableCell cell = GetCell(index);
		string value = cell.Value as string;
		return value;
	}

	public void ClickAtRow(int row)
	{
		TableCell cell = GetCell(row);
		Point topLeft = cell.Bounds.TopLeft;
		topLeft.X += 5;
		topLeft.Y += 5;
		Mouse.instance.Click(topLeft);
	}

	private TableCell GetCell(int index)
	{
		TableRows rows = table.Rows;
		TableCells cells = rows[index - 1].Cells;
		return cells[0];
	}
}

Access the grid

MainGrid is property inside MainWindow page object. On access to the property new object is instantiated. This might lead to performance issues if grid search and instantiation is slow. So, in this case, you can use Singleton design pattern. Singleton might lead to issues with old object state which will be hard to debug. It depends what your priorities are.

public class MainWindow : XamlElementContainer
{
	public static string WINDOW_NAME = "MainWindow";
	private Application app;
	private string mainPath =
		"XamlPath=/Border[0]/AdornerDecorator[0]/ContentPresenter[0]/Grid[0]/";
	public MainWindow(VisualFind find, Application application)
		: base(find)
	{
		app = application;
	}

	private MainGrid MainGrid
	{
		get
		{
			return new MainGrid(app.GetWindowByName(WINDOW_NAME));
		}
	}

	public void ClickTableAtRow(int row)
	{
		MainGrid.ClickAtRow(row);
	}

	public Verification VerifyTableCell(int index, string text)
	{
		return BaseTest.VerifyText(text, MainGrid.GetCellText(index));
	}
}

Conclusion

TestStack White is a powerful framework. It will be perfect if you can do the job without it. If you cannot you are lucky it exists.

Related Posts

Read more...

Advanced WPF automation – page objects inheritance

Last Updated on by

Post summary: re-use of page objects code through inheritance.

Inheritance is one of the pillars of object-oriented programming. It is a way to re-use functionality of already existing objects.

Reference

I’ve started a series with details of Advanced WPF desktop automation with Telerik Testing Framework and TestStack White. The sample application can be found in GitHub SampleAppPlus repository.

Abstract class

An abstract class is one that cannot be instantiated. An abstract class may or may not have abstract methods. If one method is marked as abstract then its containing class should also be marked as abstract. We have two similar windows with text box, save and cancel button that is shown on both of them. AddEditText class following Page Objects pattern. It is marked as abstract thought. It has an implementation of all three elements except TextBox_Text.

public abstract class AddEditText : XamlElementContainer
{
	protected string mainPath =
		"XamlPath=/Border[0]/AdornerDecorator[0]/ContentPresenter[0]/Grid[0]/";
	public AddEditText(VisualFind find) : base(find) { }

	protected abstract TextBox TextBox_Text { get; }
	private Button Button_Save
	{
		get
		{
			return Get<Button>(mainPath + "Button[0]");
		}
	}
	private Button Button_Cancel
	{
		get
		{
			return Get<Button>(mainPath + "Button[1]");
		}
	}

	public void EnterText(string text)
	{
		TextBox_Text.Clear();
		TextBox_Text.User.TypeText(text, 50);
	}

	public void ClickSaveButton()
	{
		Button_Save.User.Click();
		Thread.Sleep(500);
	}

	public void ClickCancelButton()
	{
		Button_Cancel.User.Click();
	}
}

Add Text page object

The only thing we have to do in Add Text window is to implement TextBox_Text property. All other functionality has already been implemented in AddEditText class.

public class AddText : AddEditText
{
	public static string WINDOW_NAME = "Add Text";
	public AddText(VisualFind find) : base(find) { }

	protected override TextBox TextBox_Text
	{
		get
		{
			return Get<TextBox>(mainPath + "TextBox[0]");
		}
	}
}

Edit Text page object

In Edit Text page object we have to implement “TextBox_Text” property. Also on this window, there is one more element which needs to be defined.

public class EditText : AddEditText
{
	public static string WINDOW_NAME = "Edit Text";
	public EditText(VisualFind find) : base(find) { }

	private TextBlock TextBlock_CurrentText
	{
		get
		{
			return Get<TextBlock>(mainPath + "TextBlock[0]");
		}
	}

	protected override TextBox TextBox_Text
	{
		get
		{
			return Get<TextBox>(mainPath + "TextBox[1]");
		}
	}

	public Verification VerifyCurrentText(string text)
	{
		return BaseTest.VerifyText(text, TextBlock_CurrentText.Text);
	}
}

Conclusion

Inheritance is a powerful tool. We as automation engineers should use it whenever possible.

Related Posts

Read more...

Advanced WPF desktop automation

Last Updated on by

Post summary: In this series of posts I’ll expand the examples and ideas started in Automation of WPF applications series.

Telerik Testing Framework and TestStack White are powerful tools for desktop automation. You can automate almost everything with a combination of those frameworks. This series of posts will give more details how to automate more complex applications.

Reference

Code samples are located in GitHub SampleAppPlus repository. Telerik Testing Framework requires installation as it copies lots of assemblies in GAC.

SampleAppPlusThere is SampleAppPlus which is actually a dummy application with only one purpose to be used to demonstrate automation principles. With this application, you can upload an image file. Once uploaded image is visualized. The image path is listed in a table. The image path is also visualized as an image in a custom control in the bottom of the main window. The user is able to add more text which is added to the table as long as editing already existing text. Add and edit are reflected on custom image element.

Topics

  • Page objects inheritance of similar windows
  • Working with WinForms grid
  • Windows themes and XamlPath
  • Read dependency property
  • NTestsRunner in action
  • Extension methods
  • Memory usage

Page objects inheritance

It is common to have similar windows in an application. Each window is modeled as page object in automation code. If windows are also similar in terms of internal structure it is efficient to re-use similar part and avoid duplications. Re-use is achieved with inheritance. Given SampleAppPlus application has very similar windows for adding and editing text. Code examples show how to optimize your effort and re-use what is possible to be re-used. More details can be found in Advanced WPF automation – page objects inheritance post.

Working with WinForms grid

As mentioned before Telerik Testing Framework is not very good with WinForms elements. This is the main reason to use TestStack White. It is not very likely to have WinForms elements in WPF application but in order to complete the big picture I’ve added such grid in a SampleAppPlus application. Code examples show how to manage WinForms grid. More details can be found in Advanced WPF automation – working with WinForms grid post.

Windows themes and XamlPath

In given examples elements are located with exact XamlPath find expression. This approach has a serious problem related to Windows themes. For complex user interfaces, XamlPath could be different on a different theme. Windows Classic theme sometimes produces different XamlPath in comparison with standard Windows themes. Yes, it is no more available from Windows 8 but Server editions are working only with Windows Classic theme. So one and the same tests could have differences. I couldn’t find a way to automatically detect which is current theme. The solution is to have different XamlPath for both standard and classic themes. Once you have it you can switch them manually with some configuration or you can try to automate the switch by locating element for which you know is different and save variable based on its location result.

Read dependency property

A dependency property is a way in C# to extend the standard provided functionality. It can happen in a real application that developers use such functionality. Given SampleAppPlus application has a special element with dependency property. Code examples show how to extract property value and use it in your tests. More details can be found in Advanced WPF automation – read dependency property post.

NTestsRunner in action

I’ve introduced NTestsRunner which is a custom way for running functional automated tests. Code samples show how to use it and create good tests that are run only with this tool.

Extension methods

Extension methods are one extremely good feature of .NET framework. I personally like them very much. I assume everyone writing code in C# is aware of them. Still, in code examples show how they can be used.

Memory usage

Memory is not a problem on small projects. But when the number of tests continue to grow it actually becomes a problem. More details can be found in Advanced WPF automation – memory usage post.

Related Posts

Read more...

Multilingual automation testing with enumerations

Last Updated on by

Post summary: Solution for automated testing of multilingual sites by using string values in all supported languages for enumerations.

In efficiently use of enumerations with string values in C# post I’ve described how you can add text to an enumeration element and then use it. Current post is elaboration with code samples for testing multilingual applications.

The challenge

Multilingual automation is always a challenge. If you use text to locate elements or verify condition then trying to run a test with different language will fail. Enumerations with language dependent string values is a pretty good solution. How to do it is described below.

Define attribute

StringValue class is extending System.Attribute. It has two properties for text and language. It should have AllowMultiple = true in order to be applied as many times as many languages you have.

namespace System
{
	[AttributeUsage(AttributeTargets.Field, AllowMultiple = true)]
	public class StringValue : Attribute
	{
		public string Value { get; private set; }
		public string Lang { get; private set; }

		public StringValue(string lang, string value)
		{
			Lang = lang;
			Value = value;
		}
	}
}

Read attribute

With reflection read all StringValue attributes. Iterate them and return the one that matches language given as parameter.

using System.Reflection;

namespace System
{
	public static class ExtensionMethods
	{
		public static string GetStringValue(this Enum value, string lang)
		{
			string stringValue = value.ToString();
			Type type = value.GetType();
			FieldInfo fieldInfo = type.GetField(value.ToString());
			StringValue[] attrs = fieldInfo.
				GetCustomAttributes(typeof(StringValue), false) as StringValue[];
			foreach (StringValue attr in attrs)
			{
				if (attr.Lang == lang)
				{
					return attr.Value;
				}
			}
			return stringValue;
		}
	}
}

Apply to enumerations

All supported languages can be defined as string constants. It will be pretty cool if can define an enumeration with languages and pass it in the StringValue constructor as a language but it is not possible as it is not a compile-time constant.

public class Constants
{
	public const string LangEn = "en";
	public const string LangFr = "fr";
	public const string LangDe = "de";
}

public enum Messages
{
	[StringValue(Constants.LangEn, "Problem occured, try again later")]
	[StringValue(Constants.LangFr, "Problème survenu, réessayer plus tard")]
	[StringValue(Constants.LangDe, "Problem aufgetreten, " +
		"versuchen Sie es später erneut")]
	ProblemOccured,
	[StringValue(Constants.LangEn, "Successfully done")]
	[StringValue(Constants.LangFr, "Fait avec succès")]
	[StringValue(Constants.LangDe, "Erfolgreich durchgeführt")]
	Success
}

Use in code

Somewhere at a top level of your tests, you should have property or field which most likely will be read from conflagration and will define for which locale is the current test run.

string lang = Constants.LangFr;

This is then used to read correct text value for given enumeration element.

Assert.AreEqual(Messages.ProblemOccured.GetStringValue(lang), 
	App.MessageBox.GetText());

Conclusion

Multilingual testing is a challenge. Be smart and use all tricks you might get. In this post, I’ve revealed pretty good trick to do the automation. Challenge with this approach will be initially set up of enumerations with all the translations.

Related Posts

Read more...

Efficiently use of enumerations with string values in C#

Last Updated on by

Post summary: Using enumerations or specialized classes makes your automation tests easy to understand and maintain. Show with code samples how to define and read string value to enumeration elements.

When you do automation tests and have to pass a value to a method it is so easy and natural to just use strings. There are many cases where a string is a correct solution. There are also many cases where a string can be a solution, but enumeration or specialized class are better and more efficient solution.

Why not strings

Having the following example – web application with drop down which has several options. We are using Page objects pattern to model the page. Page object has a method which accepts the option to be selected. String seems like a natural solution but is wrong. Although string will work enumeration is the only right solution. Drop down has limited and already defined options that can be selected. Exposing just string may cause misinterpretations for the consumer of your method. It is much more easy to limit the consumer to several enumeration values. In this way, consumer knows what data to provide and this automatically keeps code clean from magic strings. If changes are needed they will be done only in the enumeration making code easier to maintain.

Problem with enumerations in C#

Using enumerations for example given above will not work. Unlike Java enumerations in C# are wrappers for int or other numeric types value. You are not able to use text with enumeration element.

Using string values with enumerations

Only way to use string values in enumerations is by adding it as an attribute to each enumeration’s element. It takes several steps in order to accomplish this.

  1. Create the attribute class that will be applied to enumeration element
  2. Create extension method that is responsible for reading a string value from enumeration element
  3. Apply string value attribute to enumeration element
  4. Use in code

Below are code samples how to use string values with enumerations in C#. Defining and reading of the attribute is functionality built in NTestsRunner.

Define attribute

The first step is to create a class that extends System. Attribute. It has only one string property to hold the text in it. The text is passed in the constructor. Note that this class is defined in System namespace in order to have it by default skipping the need of importing namespace you might not be aware of.

namespace System
{
	public class StringValue : Attribute
	{
		public string Value { get; private set; }

		public StringValue(string value)
		{
			Value = value;
		}
	}
}

Read the attribute

C# provides so-called extension methods, a great way to add new functionality to the existing type without creating new derived type. Reading of string value from enumeration element is done with a GetStringValue extension method. With reflection, all StringValue custom attributes of an element are obtained. If some found text of first is returned. If not then string representation of the element is returned.

using System.Reflection;

namespace System
{
	public static class ExtensionMethods
	{
		public static string GetStringValue(this Enum value)
		{
			string stringValue = value.ToString();
			Type type = value.GetType();
			FieldInfo fieldInfo = type.GetField(value.ToString());
			StringValue[] attrs = fieldInfo.
				GetCustomAttributes(typeof(StringValue), false) as StringValue[];
			if (attrs.Length > 0)
			{
				stringValue = attrs[0].Value;
			}
			return stringValue;
		}
	}
}

Apply to enumerations

Once StringValue class is ready it can be applied as an attribute to any enumeration.

public enum Messages
{
	[StringValue("Problem occured, try again later")]
	ProblemOccured,
	[StringValue("Successfully done")]
	Success
}

Use in code

In code string value can be obtained from enumeration’s element with a GetStringValue method.

Assert.AreEqual(Messages.ProblemOccured.GetStringValue(), App.MessageBox.GetText());

Conclusion

Using enumerations is mandatory to make readable and maintainable automation. Working effectively with enumerations will increase your value as automation specialist.

Related Posts

Read more...

NTestsRunner for functional automated tests

Last Updated on by

Post summary: NTestsRunner implementation details and features.

In the previous post I’ve described unit testing frameworks and why they are not suitable for running functional automated tests. I introduced NTestsRunner – very simple runner that can be used for running your automation tests. This topic is dedicated to implementation details of the NTestsRunner.

Verifications

It is important in functional testing to be able to place several verification points in one test. For this purpose, abstract class Verification is implemented. It has two properties to store more details about verification and time it was taken. Constructor receives comma separated string values. In case of zero strings are passed then the result is an empty string. If one string is passed then this is the result. If more than one string is added then first string is taken as formatting string and others are used to build up the result. Logic is similar to string.Format(String, Object[]) method.

public abstract class Verification
{
	public string Result { get; private set; }
	public DateTime ExecutedAt { get; private set; }

	public Verification(params object[] args)
	{
		...
	}
}

Passed or Failed

In automation test may have two conditions – passed or failed. This is why two concrete classes are extending Verification: VerificationPassed and VerificationFailed. They do not add any other functionality. Those classes use parent’s class constructor. This is an example how to instantiate an object from those classes:

string value = "number";
int number = 1;
Verification result =
	new VerificationFailed("This is formatting string {0} {1}. ",
		value,
		number);

Test case result

Test case is generally a set of conditions to verify whether given scenario works are per user requirements. In automation, world test case is test method with several verification points inside. In NTestsRunner TestCaseResult is class representing the idea of a test case. It has properties for name, time to run and list of all verifications with a count of passed and failed.

public class TestCaseResult
{
	public List<Verification> Verifications = new List<Verification>();
	public string Name { get; set; }
	public int VerificationsFailed { get; set; }
	public int VerificationsPassed { get; set; }
	public TimeSpan Time { get; set; }
}

Test plan result

TestPlanResult in NTestsRunner has nothing to do with test plan term from QA world. Here this is a representation of a test class with test methods inside. It has properties for name and time to run. Also, there is a list of all TestCaseResults, i.e. test methods in that class. There are counters for passed and failed test cases and also counters for all passed and failed verifications inside all TestCaseResults.

public class TestPlanResult
{
	public List<TestCaseResult> TestCases = new List<TestCaseResult>();
	public string Name { get; set; }
	public int TestCasesPassed { get; private set; }
	public int TestCasesFailed { get; private set; }
	public int VerificationsPassed { get; private set; }
	public int VerificationsFailed { get; private set; }
	public TimeSpan Time { get; private set; }

	public void Count()
	{
		...
	}
}

Class and method attributes

In order to make one class a test class, it should have with [TestClass] attributes. To convert method to a test one it should have [TestMethod] attribute. Just the attribute is not enough though. The method should have special method signature. This is required by NTestsRunner.

Test method signature

In order to run without exception test method needs to conform to two rules:

  1. To have attribute [TestMethod]
  2. Method to receive parameter List verifications in its signature, i.e.
    [TestMethod]
    public void TestMethod1(List<Verification> verifications)
    

Configurations

Configurations can be found on NTestRunner home page.

Execution

Once object from NTestsRunner is instantiated and configured tests with Execute() method. Inside this method, all classes from calling assembly (the one that holds the tests) are taken. If TestsToExecute is configured then only those with name matching is given values are taken. If no TestsToExecute is provided then all classes with the attribute [TestClass] are taken. Methods from each class are taken by default in order of appearance in the class. If the method has [TestClass] attribute then the method is executed by passing List object to it. Inside the method, Verifications are collected as a list into a TestCaseResult object. After the method is run TestCaseResult is added to its parent TestPlanResult which is added to list with all results. In the end, results are saved as XML and HTML.

Results in JUnit XML

In order to integrate with CI tools such as Jenkins or Bamboo results are exported to XML file after execution has finished. The file is named Results.xml and is located in test results folder. XML format is implemented according to junit-4.xsd.

Results in HTML

Tests result are saved as HTML report for better readability. The file is named Results.html and is located in test results folder.

Usage

In order to use NTestsRunner a console application project is needed. This project will hold test classes. As the one below. Take into consideration that this is very simplified usage pattern. In reality, Page objects design pattern will be used. Page objects will make the verifications and return them.

[TestClass]
public class TestClass1
{
	[TestMethod]
	public void TestMethod1(List<Verification> verifications)
	{
		// Do some actions
		verifications.Add(new VerificationFailed("There is error"));
		// Do some actions
		verifications.Add(new VerificationPassed("Everythign is OK"));
	}
}

In its main method, a new instance of NTestsRunner is created. Configurations are done and test executions are started. It is that simple to use it.

class Program
{
	static void Main(string[] args)
	{
		NTestsRunnerSettings settings = new NTestsRunnerSettings();
		settings.TestResultsDir = @"C:\temp";
		settings.MaxTestCaseRuntimeMinutes = 2;
		settings.TestsToExecute.Add("TestClass1");
		settings.PreventScreenLock = true;

		NTestsRunner runner = new NTestsRunner(settings);
		runner.Execute();
	}
}

Pros and cons

NTestsRunner has its pros and cons.
Pros are:

  • Pretty easy to use
  • Open source and can be customized to your specific needs
  • Gives you ability to make several verifications in one test and in case of failure it doesn’t break current test method
  • Tests are stored in console application that can be easily run
  • Results are saved in JUnit XML for CI integration
  • Results are saved in HTML

Cons are:

  • Test methods should have a specific signature
  • It is not easy to migrate existing tests to a new format

Conclusions

This is a pretty good tool for running functional automated tests. It is very easy to use and is made especially for running functional automated tests. You can definitely give it a try.

Related Posts

Read more...

Running functional automation tests

Last Updated on by

.Post summary: Unit testing frameworks are not very suitable for running functional tests. NTestsRunner is an alternative way of running functional automated tests.

Unit testing

Unit testing is focused on testing of code on a low level. Methods, sets of methods or modules are being tested by writing test code which invokes those methods with specific arguments. In unit testing, all external dependencies (database, file system, network, etc.) are removed. Those resources are simulated in unit tests by using so-called mock objects. Mock objects are controlled by tests designer and have predictive behavior. Running a piece of code which doesn’t have external dependencies happens almost immediately. So unit tests are executed for a very low amount of time. It is considered a set of unit tests taking longer than 5 minutes is not well designed. Unit tests are strictly focused. One test tests only one condition. Each test is not related in anyhow to other tests.

Unit testing frameworks

Unit testing frameworks conform to unit tests purpose and design. Tests should not depend on each other. For this reason, unit testing frameworks execute tests in random order (xUnit.net or MS Unit Testing Framework), other like NUnit in alphabetic order of tests method names. Checking for given conditions are done with assertions. If one assert fails current test execution is stopped and the test is marked as failed.

Functional testing

Functional testing is focused on ensuring that software product works as per user requirements. Real life software has external dependencies. Most software products have some kind of user interface. Automation tests are focused on verifying that UI works correctly. Transferring and rendering data to UI takes time, database operations take time, file and network operations also take time, etc. In general functional tests are more complex and take much longer to execute. In order to be efficient checking of several conditions are defined in each test. Functional tests can be manual and automated. In current post when I mention functional tests I mean only automated.

A requirement for running functional test – many verifications

In a perfect situation, a functional one test should verify only one condition. In reality, because of too many external dependencies, time for tests execution is large. A time matters. In order to shorten this time we make several checks in one test. For e.g. in an e-commerce website, an order is placed. We verify order confirmation page that there is order number, that address is same used during checkout, that user’s email is correct. We also can verify inside user’s email box that received mail is correct. We can check in a database for some properties of the order. If we have to do one order for each check tests will take significantly longer time. To be efficient we do all checks with one order in one test case. We need a framework which allows you to have multiple verifications in one test. Furthermore, if verification is failed test execution should continue.

A requirement for running functional test – controlled sequence

What is really good to be avoided but sometimes cannot is test dependency. Sometimes one test needs another to have done something before continuing. As I said this should be avoided but in order to be efficient you should make a trade-off between good tests design and time to execute. For e.g. you may want to cancel and then refund order placed in an e-commerce website. Generally, it is best to place a new order for this test but if the placing of order requires too much time then an option is to reuse already existing order from the previous test. We need to be able to control test case execution order.

NTestsRunner

In order to have control over tests and use many verifications, I’ve created NTestsRunner. Its code is in GitHub NTestsRunner repository. This is a .NET library. You create a console application with your tests and use the library within. Tests are annotated is similar fashion as with unit testing frameworks. Tests are executed in sequential order. There could be many verifications. Results are saved as HTML and XML in JUnit format. NTestsRunner is described in more details in NTestsRunner for functional automated tests post.

Related Posts

Read more...

WPF automation – running the tests

Last Updated on by

Post summary: How to sell your automation to management. Guide for running the tests unattended.

References

This post is part of Automation of WPF applications with Telerik Testing Framework and TestStack White series. The sample application can be found in GitHub SampleApp repository.

Test frameworks need mouse and keyboard

As you have noticed when running the examples it is not possible to do anything else while tests are running because both Telerik Testing Framework and TestStack White are using mouse and keyboard in order to click and type text. This is how both frameworks are designed and work. It doesn’t seem very effective if you are wasting your time watching tests running on your workstation instead of doing something productive. This is not a good argument when advocating your automation.

The essence of automation is being effective

This topic is not about Automation vs. Manual testing so I will not go in that direction. I’ll just say that still there are companies lacking management willpower to support and embrace automation. So we need to be good salesmen!

How to drive your automation to success

As I said we need to be wise when promoting our automation to get more time and resources. Sales strategy is pretty easy and straightforward not requiring huge investments:

  • Free to use and pretty easy to work with frameworks.
  • Some enthusiasm to make the first automation. Remember, be smart and first automate most repeating scenarios. This will show real results and help you buy some time for further automation.
  • Virtual (or real) machine with test running on it during the night
  • Mail with results is sent to management with results in the morning

Run the tests unattended

You get the machine, set up the framework, set up scheduled tasks for deploying the latest application under test and latest test code. Run the tests and get into trouble. Tests do not run! This is because there are special requirements to schedule unattended run. There must be an active Windows session in order mouse and keyboard to be used. Once the session is interrupted tests stop. This is detailed KB article on the topic with several possible solutions.

Working solution

You can try solutions in the article to see which works best for you. For me, the solution is to have a remote desktop in remote desktop. This requires Windows Server installation. The only server provides two simultaneous remote desktop sessions. There are unofficial patches for non-server versions which I haven’t tried and cannot comment. Two local users are needed on the Windows Server with no desktop locking or screen saver (domain users most likely will have desktop locked after a time). Log from your machine to test machine with the first user. From first user’s session log to same testing machine with the second user. Tests are started from second user’s session. Once tests are started you can freely close desktop (not log out!). You can create a scheduled task which runs only when the user is logged in and just wait for results in the morning. If two accounts are overhead there is an option to use software that prevents your computer to lock. See prevent screen lock thread if something works for you.

Conclusions

Automation is an exciting field of career development for test engineers. This blog is dedicated to automation testing. You will find very useful and interesting topics in it. I would definitely encourage you just to give it a try. Good luck!

Related Posts

Read more...

WPF automation – using the elements

Last Updated on by

Post summary: Use already created Page Objects and build up test framework.

References

This post is part of Automation of WPF applications with Telerik Testing Framework and TestStack White series. The sample application can be found in GitHub SampleApp repository.

Page Objects holder

Below is App.cs which is a representation of application under test.

using ArtOfTest.WebAii.Wpf;
using White.Core;
using White.Core.UIItems.WindowItems;

namespace SampleApp.Tests.Framework.Elements
{
	public class App
	{
		public WpfApplication ApplicationWebAii { get; private set; }
		public Application ApplicationWhite { get; private set; }

		public App(WpfApplication webAiiApp, Application whiteApp)
		{
			ApplicationWebAii = webAiiApp;
			ApplicationWhite = whiteApp;
		}

		public MainWindow MainWindow
		{
			get
			{
				return new MainWindow(ApplicationWebAii
					.WaitForWindow(MainWindow.WINDOW_NAME).Find);
			}
		}

		public OpenFile OpenFile
		{
			get
			{
				return new OpenFile(GetWindowByName("Open"));
			}
		}

		public MessageBox MessageBox
		{
			get
			{
				return new MessageBox(GetWindowByName(""));
			}
		}

		private Window GetWindowByName(string windowName)
		{
			// Workaround as method GetWindow(string title) is not working
			foreach (Window window in ApplicationWhite.GetWindows())
			{
				if (windowName.Equals(window.Name))
				{
					return window;
				}
			}
			return null;
		}
	}
}

The constructor takes an instance of Telerik Testing Framework’s application (WpfApplication) and TestStack White’s application (Application). Those are stored in the App instance.

Access the Page Objects

Each window in a real application is represented by a property in the App class. When accessed a new object of this page object class is created and its elements can be accessed.

WPF page objects require VisualFind in order to be instantiated. It is obtained by first locating the window with Telerik’s

public WpfWindow WaitForWindow(string caption);

From the located window we need only the VisualFind which is used internally to locate elements on that particular window.

WinForms page objects require White’s Window instance in order to be instantiated. The window is located by

public virtual Window GetWindow(string title);

I found this method not always working so I’ve made a workaround method

private Window GetWindowByName(string windowName);

New Page Objects vs. Cached Page Objects

In the example above every time, an action is required a new page object is instantiated. In some cases instantiating the object may require longer time or you might need some properties in this object preserved during tests. In such cases, you may use Singleton design pattern and instantiate only one object.

private MessageBox messageBox = null;
public MessageBox MessageBox
{
	get
	{
		if (messageBox == null)
		{
			messageBox = new MessageBox(GetWindowByName(""));
		}
		return messageBox;
	}
}

Both approaches have pros and cons. In case of new page object, you always work with a fresh instance without any previous state saved. This might require more time to instantiate the objects and you are not able to save previous states. Cached objects may be much faster as a performance but having internal state may lead to unexpected bugs in your automation.

Base test

Finally to make all work we need an instance of App. The instance is created in BaseTest.cs class.

using ArtOfTest.WebAii.Core;
using SampleApp.Tests.Framework.Elements;
using White.Core;

namespace SampleApp.Tests.Framework.Tests
{
	public class BaseTest
	{
		protected App App { get; set; }
		private string applicationPath =
			"C:\\SampleApp\\SampleApp\\bin\\Debug\\SampleApp.exe";

		protected void Start()
		{
			if (App == null)
			{
				Application appWhite = Application.Launch(applicationPath);
				Manager manager = new Manager(false);
				manager.Start();
				App = new App(
					manager.ConnectToApplication(appWhite.Process), appWhite);
			}
		}

		protected void Stop()
		{
			if (App != null && App.ApplicationWhite != null)
			{
				App.ApplicationWhite.Kill();
			}
			App = null;
		}
	}
}

All tests inherit from the base test class. Initialise and clean up code is added in the base test. In our case Start() method is the initializer. It must be called in order to instantiate App class. App property is protected so every extending class has access to it.

Initialise the frameworks

In order to start the application under test, we need the full path to the exe file. In this example, this is hardcoded but in real life, it will be configurable. Start the application with White:

public static Application Launch(string executable);

Once started then connect to it with Telerik framework by creating a Manager and use its

public WpfApplication ConnectToApplication(Process proc, string pid = null);

The process is obtained out of White Application.Process property. Opposite launch order is not working. White is not able to Attach to running process.

Use page objects

Once Start() method is called, the application under test is started and both frameworks are connected to it you can simply do in your test:

App.MainWindow.ClickBrowseButton();

This will find and create a new instance of MainWindow and then it will find and click Browse button. Your framework defines the actions on elements which are later used in actual tests. Once all the work on framework has been done it is that simple to build your tests.

Clean up

Stop() method is called at the end of the test in order to close the application under test by killing the underlying process.

The tests

This is unit test created with MS Unit Testing Framework in order to demonstrate real testing on the application.

using Microsoft.VisualStudio.TestTools.UnitTesting;
using SampleApp.Tests.Framework.Tests;

namespace SampleApp.Tests
{
    [TestClass]
    public class UnitTest1 : BaseTest
    {
		[TestInitialize]
		public void Initialise()
		{
			Start();
		}

		[TestMethod]
		public void OpenFile_OnCancel_GivesMessage()
		{
			App.MainWindow.ClickBrowseButton();
			App.OpenFile.ClickCancelButton();
			Assert.AreEqual("Problem occured, try again later",
				App.MessageBox.GetText());
			App.MessageBox.ClickOkButton();
		}

		[TestMethod]
		public void OpenFile_OnAttachFile_GivesMessageAndFileIsShown()
		{
			string filePath = @"C:\SampleApp\SampleApp\bin\Debug\HappyFace.jpg";
			App.MainWindow.ClickBrowseButton();
			App.OpenFile.EnterFileName(filePath);
			App.OpenFile.ClickOpenButton();
			Assert.AreEqual("Successfully done", App.MessageBox.GetText());
			App.MessageBox.ClickOkButton();
			Assert.AreEqual(filePath, App.MainWindow.GetFilePathAtIndex(1));
		}

		[TestCleanup]
		public void CleanUp()
		{
			Stop();
		}
    }
}

Unit testing frameworks

Unit testing frameworks are designed to run tests in random order. Before each test method annotated with [TestInitialize] is run. In our case application is started. After each test method annotated with [TestCleanup] is run. In our case application is stopped. For this simple application running tests with a unit testing framework is OK. We are not doing unit tests but functional once. So for bigger and more complex tests unit testing frameworks are not very convenient. I’ve created very simple tests runner. This post describing the need of such tests runner.

This post shows how to build up the framework based on page objects. Next post is WPF automation – running the tests.

Related Posts

Read more...

WPF automation – locating and structure of WinForms elements

Last Updated on by

Post summary: Guide how to locate WinForms elements with TestStack White.

References

This post is part of Automation of WPF applications with Telerik Testing Framework and TestStack White series. The sample application can be found in GitHub SampleApp repository.

WinForms elements manipulation with TestStack White

MessageBox.cs class is a representation of a MsgBox of our sample application which is WinForms.

using System.Windows.Automation;
using White.Core.UIItems;
using White.Core.UIItems.Finders;
using White.Core.UIItems.WindowItems;

namespace SampleApp.Tests.Framework.Elements
{
	public class MessageBox
	{
		private Window win;
		public MessageBox(Window window)
		{
			win = window;
		}

		private Button Button_OK
		{
			get
			{
				return win.Get<Button>(
					SearchCriteria.ByControlType(ControlType.Button).
					AndByText("OK"));
			}
		}
		private Label Label_Text
		{
			get
			{
				return win.Get<Label>(
					SearchCriteria.ByControlType(ControlType.Text));
			}
		}

		public void ClickOkButton()
		{
			Button_OK.Click();
		}

		public string GetText()
		{
			return Label_Text.Text;
		}
	}
}

MessageBox is following Page Object design pattern. Elements are private properties. Actions are public methods acting on elements. Again: one element must be defined in only one place.

Constructor receives an instance of the current window object. Elements are located by White framework method:

public virtual T Get<T>(SearchCriteria searchCriteria) where T : UIItem;

SearchCriteria

SearchCriteria is used for locating elements. Search is possible with a combination of following criteria: Text, ControlType, AutomationId, NativeProperty, Framework. And here comes the hardness of working with White in complex UIs. There may be several TextBox-es (ByControlType == Edit) AND with no text inside (AndByText == “”). You can take them all and then access by index the one which is proper for your test. And what happens when you fill text in one of them. Next find has one less window. If you think it is easy then enjoy, I would rather not do it.

UISpy

Finding of the proper SearchCriteria is done with powerful tool UISpy.exe. Unfortunately, this one is deprecated from Windows 8. MS have added a replacement called Inspect.exe. I don’t find it useful this is why I have added UISpy.exe in the GitHub repository.

UISpy

Dialog is the message box window which is a child of MainWindow. It has empty title and two elements: OK button and message text. Current example shows the location of the text element. Its MS ControlType == ControlType.Text. This is what we need to locate it since there is only one such element in the message box. Once we have it we take the text and test compares current vs. expected text. OK button is located by ControlType and text. In case of different localisations we may leave just the ControlType, but if there are more buttons text is mandatory to distinguish the correct button.

Once you have defined your page objects it is time to assemble them. Next post is WPF automation – locating and structure of WPF elements.

Related Posts

Read more...

WPF automation – locating and structure of WPF elements

Last Updated on by

Post summary: Guide how to locate WPF elements with Telerik Testing Framework.

References

This post is part of Automation of WPF applications with Telerik Testing Framework and TestStack White series. The sample application can be found in GitHub SampleApp repository.

MainWindow Page Object

MainWindow.cs class is a representation of the main window of our sample application which is WPF. Note that code here is shortened. The full code can be found in GitHub repository.

using ArtOfTest.WebAii.Controls.Xaml.Wpf;
using ArtOfTest.WebAii.Silverlight;
using ArtOfTest.WebAii.TestTemplates;
using System.Threading;

namespace SampleApp.Tests.Framework.Elements
{
	public class MainWindow : XamlElementContainer
	{
		public static string WINDOW_NAME = "MainWindow";
		private string mainPath =
			"XamlPath=/Border[0]/AdornerDecorator[0]/ContentPresenter[0]/Grid[0]/";
		public MainWindow(VisualFind find) : base(find) { }

		private Button Button_Browse
		{
			get
			{
				return Get<Button>(mainPath + "Button[0]");
			}
		}

		public void ClickBrowseButton()
		{
			Button_Browse.User.Click();
			Thread.Sleep(500);
		}
	}
}

MainWindow is following Page Object design pattern. Elements are private Properties. Actions on elements are public methods that are the actual building blocks of the tests. Do not just add action methods for the sake of adding them. Add them if only the tests need such action.

If the element is used in only one action then it could be inside the action method, but if the element is used more than one it is mandatory to keep it as separate property. The main idea is to avoid duplications in order to improve maintainability. Remember to stay DRY: one element must be defined in only one place.

MainWindow class extends XamlElementContainer which comes from the framework. The constructor takes VisualFind object and passes it to XamlElementContainer‘s constructor. This allows us to use framework’s methods for location of elements. In our case

public TControl Get<TControl>(params string[] clauses)
	where TControl : ArtOfTest.WebAii.Controls.Xaml.IFrameworkElement;

This is the strategy used by the Test Studio generated tests and I prefer it.

A different approach for MainWindow

Another element structure could be not to extend XamlElementContainer but pass a VisualFind object through MainWindow’s constructor and use it internally to locate elements.

using ArtOfTest.WebAii.Controls.Xaml.Wpf;
using ArtOfTest.WebAii.Silverlight;
using System.Threading;

namespace SampleApp.Tests.Framework.Elements
{
	public class MainWindow
	{
		public static string WINDOW_NAME = "MainWindow";
		private string mainPath =
			"XamlPath=/Border[0]/AdornerDecorator[0]/ContentPresenter[0]/Grid[0]/";
		private VisualFind visualFind;
		public MainWindow(VisualFind find)
		{
			visualFind = find;
		}

		private Button Button_Browse
		{
			get
			{
				return visualFind.ByExpression(
					new XamlFindExpression(mainPath + "Button[0]")).
					CastAs<Button>();
			}
		}

		public void ClickBrowseButton()
		{
			Button_Browse.User.Click();
			Thread.Sleep(500);
		}
	}
}

How to locate elements is described in finding page elements article.

XamlPath

As you can see I’m using exact XamlPath find expression in order to locate my elements because I find it consistent and easy to maintain. XamlPath syntax is pretty similar to XPath. The difference is its indexes are zero-based (Button[0] is the first element) and it doesn’t provide predicates and axes. The hard part is to get the XamlPath. I have found this lovely tool WPF Inspector. It gives a lot of information about the structure of WPF application.

WPF Inspector

WPF Inspector

This is a screenshot of the XamlPath for Browse button. In more complex UIs even with this great tool, it is not that easy to get the XamlPath. If you find it hard you may consider other find expressions.

All elements on MainWindow are only WPF thus an instance of White is not needed. In case of WinForms content hosted inside WPF, you can pass an instance of White through elements’ constructor and use it in actions inside.

public MainWindow(VisualFind find, Application applicationWhite) : base(find)
{
	appWhite = applicationWhite;
}

Having an instance of White inside the class you can work with it as explained in WPF automation – locating and structure of WinForms elements post.

Related Posts

Read more...

WPF automation – projects structure

Last Updated on by

Post summary: How to structure projects in the course of WPF automation.

References

This post is part of Automation of WPF applications with Telerik Testing Framework and TestStack White series. The sample application can be found in GitHub SampleApp repository.

Overview

For better maintainability and good design, automation testing is separated into two projects. One is the so-called framework project. It has knowledge about third used party frameworks (Telerik Testing Framework, TestStack White, Selenium, Watij, etc.). It operates on elements via those third-party frameworks. Framework exposes actions that are building blocks of the tests. The other project is the tests. It doesn’t know anything about used automation frameworks. It uses methods exposed by an internal framework and builds the tests with them. The idea is that one team/person with development knowledge can be responsible for the internal framework, others can be responsible for writing well-designed tests using the internal framework only. Another benefit of two projects is the ability to completely change internal framework project without affecting test logic at all. Here is the structure of my sample projects:

Projects structure

Projects structure

SampleApp – This is the application under test. Generally, it is not in the test project, you receive it from developers or from nightly build or from some other place. For the purpose of this demo, I’ve created a very basic application that uploads an image and shows it inside. There is attached image (HappyFace.jpg) that is used in the tests.

SampleApp.Tests – Here is the most important artifact – the tests, the purpose of all this fuss. In the demo, this is made as UnitTest with MS Unit Testing Framework. You can use whatever run strategy you like – another unit testing framework (xUnit, nUnit, etc.) or you can build your own tests runner. I have built my own tests running because by definition unit tests should be independent of each other thus MS Unit Testing Framework runs them in random order. It might be possible to manage order, but this will require managing of some sort of external files. It gives too much overhead to me.

SampleApp.Tests.Framework – This is where the interesting part is. Here are the elements – main application form (WPF), message boxes shown on success or error (WinForms) and open file dialogue (WinForms). MainWindow is manipulated with Telerik Testing Framework, other two are manipulated with White. App.cs is a holder which represent application itself and elements are accessible through it. BaseTest holds an instance of App. It is extended by UnitTests and App and its elements are accessible.

Next post is WPF automation – locating and structure of WPF elements in the code.

Related Posts

Read more...

Automation of WPF applications

Last Updated on by

Post summary: Overview how to successfully automate WPF application with Telerik Testing Framework and TestStack White.

I’ve spent lots of time in automation of a WPF application. Nowadays the world is dealing with web application and desktop application are fading away. Nevertheless, desktop applications still have their own private space and I hope these series of posts will be useful to someone.

Preface

An application under test is created with WPF but has WinForms grids included on different windows presenting lots of information. Performance is the main reason to use WinForms grids inside a WPF application. WPF grid with 40 columns and 100 rows lags a lot. After research and proof of concept of commercial and open source tools and frameworks, two were chosen: Telerik Testing Framework and TestStack White. They provide flexibility and maintainability which is so vital in a fast-changing application. Both are used with C#. Every one has its own weaknesses and strengths but both made a perfect synergy to provide you the possibility to automate almost everything.

Telerik Testing Framework

Extremely powerful and free to use framework. You can purchase additional support which is at reasonable prices. Telerik also provides Test Studio, a tool and MS Visual Studio plug-in, build on top of the framework. This is also an option, but for me using a tool gives more restrictions than benefits. Their framework is the best available for testing WPF applications. It is very easy to use. Elements are easily located by XamlPath (very similar to XPath). Of course, there are other locate options. The framework provides very rich API with lots of operations over the elements. Still, its powerfulness is limited to WPF. It is not that good with WinForms elements. Telerik Testing Framework requires installation as it copies lots of assemblies in GAC.

TestStack White

Extremely powerful open source framework. Build as a wrapper of Microsoft UI Automation framework you can automate almost everything with it. But this comes at a price. It is hard to locate elements. I use it only where Telerik Testing Framework is not capable to do the work – WinForm grids, context menus, closing windows with small X in the top right corner?! Yes, those are WPF inside but are hosted in operation system window and Telerik has no access to it.

In GitHub, I have uploaded an MS Visual Studio 2013 project with very simple application taken from this example. I have added very basic tests to it to illustrate how it is done. With upcoming posts, I’ll explain in details the idea and implementation.

Below is a list of posts included in the series:

  1. Introduction (current post)
  2. WPF automation – projects structure
  3. WPF automation – locating and structure of WPF elements
  4. WPF automation – locating and structure of WinForms elements
  5. WPF automation – using the elements
  6. WPF automation – running the tests

Related Posts

Read more...

Complete guide how to use design patterns in automation

Last Updated on by

Post summary: Complete code example of how design patterns can be used in real life test automation.

With series of posts, I’ve described 5 design patterns that each automation test engineer should know and use. I’ve started with a brief description of the patterns. Then I’ve explained in details with code snippets following patterns: Page objects, Facade, Factory, Singleton, Null object. Code examples are located in GitHub for C# and Java.

Overview

This post is intended to bond all together in a complete guide how to do write better automation code. Generally, automation project consists of two parts. Automation framework project and tests project. Current guide is intended to describe how to build your automation testing framework. How to structure your tests is a different topic. Remember once having correctly designed framework then tests will be much more clean, maintainable and easy to write. To keep post shorter some of the code that is not essential for representing the idea is removed. The whole code is on GitHub.

Page objects

Everything starts by defining proper page objects. There is no fixed recipe for this. It all depends on the structure of application under test. The general rule is that repeating elements (header, footer, menu, widget, etc) are extracted as separate objects. The whole idea is to have one element defined in only one place (stay DRY)! Below is our HomePage object. What you can do generally is make search and clear search terms. Note that clearing is done with jQuery code. This is because of a bug I’ve described with a workaround in Selenium WebDriver cannot click UTF-8 icons post.

using OpenQA.Selenium;

namespace AutomationRhapsody.DesignPatterns
{
	class HomePageObject
	{
		private WebDriverFacade webDriver;
		public HomePageObject(WebDriverFacade webDriver)
		{
			this.webDriver = webDriver;
		}

		private IWebElement SearchField
		{
			get { return webDriver.FindElement(By.Id("search")); }
		}

		public void SearchFor(string text)
		{
			SearchField.SendKeys(text);
		}

		public void ClearSearch()
		{
			webDriver.ExecuteJavaScript("$('span.cancel').click()");
		}
	}
}

WebDriver factory

WebDriver factory will be responsible for instantiating the WebDriver based on a condition which browser we want to run our tests with.

using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.IE;

namespace AutomationRhapsody.DesignPatterns
{
	public class WebDriverFactory
	{
		public IWebDriver CreateInstance(Browsers browser)
		{
			if (Browsers.Chrome == browser)
			{
				return new ChromeDriver();
			}
			else if (Browsers.IE == browser)
			{
				return new InternetExplorerDriver();
			}
			else
			{
				return new FirefoxDriver();
			}
		}
	}
}

The constructor takes an argument browser type. Browser type is defined as an enumeration. This is very important. Avoid passing back and forth strings. Always stick to enums or special purpose classes. This will save you time investigating bugs in your automation.

public enum Browsers
{
	Chrome, IE, Firefox
}

NullWebElement

This is null object pattern and implements IWebElement. There is a NULL property that is used to compare is given element is not found or no.

using OpenQA.Selenium;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Drawing;

namespace AutomationRhapsody.DesignPatterns
{
	public class NullWebElement : IWebElement
	{
		private const string nullWebElement = "NullWebElement";
		public bool Displayed { get { return false; } }
		public bool Enabled { get { return false; } }
		public Point Location { get { return new Point(0, 0); } }
		public bool Selected { get { return false; } }
		public Size Size { get { return new Size(0, 0); } }
		public string TagName { get { return nullWebElement; } }
		public string Text { get { return nullWebElement; } }
		public void Clear() { }
		public void Click() { }
		public string GetAttribute(string attributeName) { return nullWebElement; }
		public string GetCssValue(string propertyName) { return nullWebElement; }
		public void SendKeys(string text) { }
		public void Submit() { }
		public IWebElement FindElement(By by) { return this; }
		public ReadOnlyCollection<IWebElement> FindElements(By by)
		{
			return new ReadOnlyCollection<IWebElement>(new List<IWebElement>());
		}

		private NullWebElement() { }

		private static NullWebElement instance;
		public static NullWebElement NULL
		{
			get
			{
				if (instance == null)
				{
					instance = new NullWebElement();
				}
				return instance;
			}
		}
	}
}

WebDriver facade

WebDriver facade main responsibility is to define custom behavior on elements location. This gives you centralized control over elements location. The constructor takes browser type and uses the factory to create WebDriver instance which is used internally in the facade. FindElement method defines explicit wait. If the element is not found then NullWebElement which is actual implementation of Null object pattern. The idea is to safely locate elements with try/catch and then just use them skipping checks for null.

using OpenQA.Selenium;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.ObjectModel;

namespace AutomationRhapsody.DesignPatterns
{
	public class WebDriverFacade
	{
		private IWebDriver webDriver = null;
		private TimeSpan waitForElement = TimeSpan.FromSeconds(5);

		public WebDriverFacade(Browsers browser)
		{
			WebDriverFactory factory = new WebDriverFactory();
			webDriver = factory.CreateInstance(browser);
		}

		public void Start(string url)
		{
			webDriver.Url = url;
			webDriver.Navigate();
		}

		public void Stop()
		{
			webDriver.Quit();
		}

		public object ExecuteJavaScript(string script)
		{
			return ((IJavaScriptExecutor)webDriver).
				ExecuteScript("return " + script);
		}

		public IWebElement FindElement(By by)
		{
			try
			{
				WebDriverWait wait = new WebDriverWait(webDriver, waitForElement);
				return wait.Until(ExpectedConditions.ElementIsVisible(by));
			}
			catch
			{
				return NullWebElement.NULL;
			}
		}
	}
}

Tests

As I mentioned initially this post is about using efficiently design patterns in your framework automation project. Tests design are not being discussed here. Once you have designed the framework one simple test (without asserts) that makes search will look like the code below.

Browsers browser = Browsers.Chrome;
WebDriverFacade webDriver = new WebDriverFacade(browser);
webDriver.Start("https://automationrhapsody.com/examples/utf8icons.html");

HomePageObject homePage = new HomePageObject(webDriver);
homePage.ClearSearch();
homePage.SearchFor("automation");

webDriver.Stop();

Conclusion

Design patterns are enabling you to write maintainable code. They increase the value of your code. As shown in this series of posts they are not so complicated and you can easily adopt them in your automation. Most important is design patterns increase your value as a professional. So make the time to learn them!

Related Posts

Read more...

Singleton and Null object patterns

Last Updated on by

Post summary: Description with code samples of Singleton and Null object design patterns.

Singleton

Singleton pattern is used in a case where you want to restrict the instantiation of a class to only one object.

More details

The class should not require parameters for its construction. If the creation of a class requires significant resources then a singleton is a suitable solution. It is good that singletons are created when they are first needed, so-called lazy initialization.

Null object pattern

The idea is to make the code simpler and safe by skipping checks for null reference of a given object:

if (someVariable != null) {
	// Do something with someVariable
}

References

This post is part of design patterns guide series. Code examples are located in GitHub for C# and Java.

More details

Null object implements given interface and its methods are doing nothing. This makes the null object more predictable. You can safely invoke methods on the null object without the threat of a NullRefferenceException to break your application. This pattern is very well combined with singleton pattern where a null object is actually a singleton. In this case, you can check for reference or equality.

Example

Most common usage of Singleton pattern is to limit WebDriver to only one object instance in the whole automation test project.

private static IWebDriver webDriver;
public static IWebDriver WebDriver
{
	get
	{
		if (webDriver == null)
		{
			webDriver = new FirefoxDriver();
		}
		return webDriver;
	}
}

I’m going to give slightly different usage example where Singleton is used along with Null object pattern. NullWebElement implements IWebElement interface so it must implement all methods and properties defined by the interface. This is done on lines 4 to 21. Properties are returning some values, but not null! Methods are doing nothing. From line 23 to 36 is the Singleton definition. If Singleton is an object you have defined then it should have a private constructor so no one is able to instantiate it. There is a private field which actually holds the reference to the singleton. NULL is a property which instantiates the singleton if not already instantiated and returns it.

public class NullWebElement : IWebElement
{
	private const string nullWebElement = "NullWebElement";
	public bool Displayed { get { return false; } }
	public bool Enabled { get { return false; } }
	public Point Location { get { return new Point(0, 0); } }
	public bool Selected { get { return false; } }
	public Size Size { get { return new Size(0, 0); } }
	public string TagName { get { return nullWebElement; } }
	public string Text { get { return nullWebElement; } }
	public void Clear() { }
	public void Click() { }
	public string GetAttribute(string attributeName) { return nullWebElement; }
	public string GetCssValue(string propertyName) { return nullWebElement; }
	public void SendKeys(string text) { }
	public void Submit() { }
	public IWebElement FindElement(By by) { return this; }
	public ReadOnlyCollection<IWebElement> FindElements(By by)
	{
		return new ReadOnlyCollection<IWebElement>(new List<IWebElement>());
	}

	private NullWebElement() { }

	private static NullWebElement instance;
	public static NullWebElement NULL
	{
		get
		{
			if (instance == null)
			{
				instance = new NullWebElement();
			}
			return instance;
		}
	}
}

There are two main benefits of code above. The first benefit comes from the Null object pattern. You can locate an element which doesn’t exist and call some of its methods without your tests to crash. You might say that comparing for null is not a big overhead, but in big having a null check before each action is a waste of time.

IWebElement element = null;
try
{
	element = webDriver.FindElement(By.CssSelector("notExisting"));
}
catch
{
	element = NullWebElement.NULL;
}
element.Click();

It might be a discussion whether it is better to silent this failure error with a null object or leave to a massive crash. My opinion is you must fail the test and if you put some logging into Click method of the NullWebElement then you can easily trace the issue. The real benefit is you will have only one failed test instead of the whole bunch failed because of the crash.

The second benefit comes from the Singleton pattern. You can easily compare some element against the NullWebElement.NULL. Most likely you will locate the element and use it safely because of null object pattern, but there might a case where you want to see if the element is actually not found.

if (element == NullWebElement.NULL)
{
	Console.WriteLine("Element not found!");
}

Conclusion

Singleton is a pattern that you definitely should know. A null object with a combination of Singleton can decrease the amount of code you write by skipping the checks for null. I would say if adopted those can simplify your code which is a benefit in big projects.

Related Posts

Read more...