Web services performance testing: a pilgrim’s progress. Part 4.

I have SoapUI Pro from SmartBear at my disposal at work, and I’m quite comfortable with it. So it was my tool of choice for creating load and recording details about the results.

SoapUI’s first cousin LoadUI, a free version of which comes with the SoapUI Pro install, was also an option. However, I chose not to explore it for this testing mission. (A road not travelled, admittedly.)

My first load tests sent requests to the middleware Web services described in Part 3 of this series. Because of our NTLM proxy at work I had to use Fiddler to do the handshake between my desktop and the Web services.

Fiddler records a lot of interesting information about Web service requests, including response times and bytesize. So I copied that data from Fiddler, pasted it into Excel, and created time-series graphs from that data. I was able to create some pretty graphs, but copying and pasting the data over and over got to be a real time sink. I am not and will never be a VBA whiz, so I knew I had to find a better way.

I was forced into finding that better way when it came time to test the performance of the .NET services that USED the middleware Web services. Because of the .NET Web server’s configuration, I could no longer route requests through Fiddler and see the data. What seemed to be a hindrance turned out to be a blessing in disguise.

The solution I arrived at was to use SoapUI to record several aspects of the request and response transactions to a flat file. I could then bring that flat file into R for graphing and analysis.

The SoapUI test case for the .NET services is set up as follows. Apologies for the blurriness of some images below: I haven’t done HTML-based documentation in quite some time.

1. Initial request to get the service “warmed up.” I do not count those results.
2. Multipurpose Groovy script step.

// I am registering the jTDS JDBC driver for use later in the test case. See below for info on using third-party libraries.

import groovy.sql.Sql
com.eviware.soapui.support.GroovyUtils.registerJdbcDriver( "net.sourceforge.jtds.jdbc.Driver" )

//Get the time now via Joda to put a time stamp on the data sink filename.

import org.joda.time.DateTime
import org.joda.time.LocalDate

def activeEnvironment = context.expand( '${#Project#activeEnvironment}' )

def now = new DateTime().toString().replaceAll("[\\W]", "_")

// Construct the file name for response data and set the filename value of a Data Sink set further along in the test.

testRunner.testCase.getTestStepByName("Response Size and Time Log").getDataSink().setFileName('''Q:/basedirectory/''' + activeEnvironment + '''_''' + now + '''_responseTimes.csv''')

If your Groovy script uses third-party libraries like jTDS and Joda, you have to put the jar files into $soapui_home/bin/etc.



Note that jTDS has an accompanying DLL for SQL Server Windows-based authentication. DLLs like this go in $soapui_home/bin.



This is how you set an activeEnvironment variable: setup happens at the project level:



Then you choose your environment at the test suite level.



3. Run a SQL query in a JDBC DataSource step to get a random policy number from the database of your choice on the fly. You can create your SQL query for the DataSource step dynamically in a Groovy script.

// State Codes is a grid Data Source step whose contents aren't shown here.

def stateCode = context.expand( '${State Codes#StateCode}' )

testRunner.testCase.getTestStepByName("Get A Policy Number").getDataSource().setQuery("SELECT top 1 number as PolicyNumber FROM tablename p where date between '1/1/2012' and getdate() and state = '" + stateCode + "' order by newid()")

4. Here’s the JDBC data source for the policy numbers.The query (not shown) is fed over from the preceding Groovy script step.



You will have to set your connection properties. Your connection string for jTDS might look something like this. For more information about jTDS, see the online docs.

jdbc:jtds:sqlserver://server:port/initialDatabase;domain=domainName;
trusted_connection=yes


5. I feed the value of PolicyNumber returned by the SQL query to my SOAP request via a property transfer.
6. I have a few assertions in the SOAP request test step. The first two are “canned” assertions that require no scripting.


The third SOAP request assertion, which is more of a functional script than it is an assertion, captures the timestamp on the response as well as the time taken to respond. These are built-in SoapUI API calls. The properties TimeStamp and ResponseTime are created and initialized in this Groovy script – I didn’t have to create them outside the script (for example, at the test step level).

import org.joda.time.DateTime

targetStep = messageExchange.modelItem.testStep.testCase.getTestStepByName('Response Size and Time Log')
targetStep.setPropertyValue( 'TimeStamp', new DateTime(messageExchange.timestamp).toString())
targetStep.setPropertyValue( 'ResponseTime', messageExchange.timeTaken.toString())

7. Another Groovy script step to get the size in bytes of the response:

def responseSize = context.expand( '${Service#Response#declare namespace s=\'http://schemas.xmlsoap.org/soap/envelope/\'; //s:Envelope[1]}' ).toString()

responseSize.length()

8. Yet another Groovy script step to save responses as XML in a data sink, with policy number, state code, and active environment as the filename:

def policyNumber = context.expand( '${Service#Request#declare namespace urn=\'urn:dev.null.com\'; //urn:Service[1]/urn:request[1]/urn:PolicyNumber[1]}' )

def stateAbbrev = context.expand( '${Service#Response#declare namespace ns1=\'urn:dev.null.com\'; //ns1:Service[1]/ns1:Result[1]/ns1:State[1]/ns1:Code[1]}' )

def activeEnvironment = context.expand( '${#Project#activeEnvironment}' )

testRunner.testCase.getTestStepByName("DataSink").getDataSink().
setFileName('''B/basedirectory/''' + policyNumber + '''_''' + stateAbbrev
+ '''_''' + activeEnvironment + '''.xml''')

9. Some property transfers, including a count of policy terms (see part 3 of this series) in the response:



10. DataSink to write the response XML to a file. The filename is set in step 7 above.



11. DataSink to write information relevant to performance to a CSV whose name is set on the fly. The filenames and property values come from the steps above.



12. Don’t forget a delay step if you’re concerned about overloading the system. (Note that you can configure delays and threading more precisely in LoadUI. This test case, like all SoapUI test cases that aren’t also LoadTests, is single-threaded by default.)



13. And of course a DataSource Loop.



14. The whole test case looks like this:



15. And now I have a CSV with response times, bytesizes, policy risk states, and term counts to parse in the tool of my choice. I chose R. More about that later.

Easy peasy emails from SoapUI test cases to you

As a sophomore-level programming autodidact, I’m on an ongoing quest to bootstrap my test automation with SmartBear‘s venerable SoapUI. You can script SoapUI as heavily as you want to with the Groovy programming language. Or you can use SoapUI’s built-in GUI elements to reduce your programming work. For example, it can be a simple matter to let SoapUI consume your project WSDL and build out a request for you… and then display the incoming response in an easy-to-read form. Or you can use groovy-wslite to start you off on the same road, but it may take you longer to write the code yourself and might not yield you any richer results.

I really wanted SoapUI to email me a simple text message when a test ended. I’d already written and tested a Groovy class that used Apache Commons’ multipart email capabilities. However, I wasn’t sure how to use that class in SoapUI. After some Googling and experimenting, here’s how I got the whole thing working today.

1. I pointed SoapUI’s script library to the folder that contained my .groovy file with the class definition.* The sixth entry on the right in the image below takes a folder location.
   
2. I dropped the Apache commons.email jar into $SoapUI_home/bin/ext, otherwise known as the bin/ext directory in your SoapUI installation folder.  (Hat tip to Saurabh Gupta for this pointer.) I would imagine that putting it into $SoapUI_home/lib would work just as well, since that’s where the SoapUI installer puts a lot of the other Apache libraries.
3. For this test case, I wanted to record all my pass/fails and email myself at the end. So I put the following code into the setup script for the test case. The setup script window is visible at the bottom of the test case GUI in SoapUI.

   context.scriptResultsList = [] // List to hold pass/fails
   context.email // to be initialized later

4. Later in the test case, a Groovy script checks one XML file against another and records “PASS” if they’re identical, “FAIL” if not, and adds the “PASS” or “FAIL” string as a list item to the scriptResultsList context variable defined in the test case setup script.

   if (xmlDiff.identical()) {
      scriptResult = 'PASS'  
    }
    else {
      scriptResult = 'FAIL'
   }
   context.scriptResultsList << scriptResult

5. In the teardown script for the test case, I call my email class by attaching it to the context.email variable I created in the setup script. I send an email whose text depends on whether any of my test cases failed. I could attach a results file with a little more work.

   if ( context.scriptResultsList.find {it == 'FAIL'} ) {
   context.email = new ApacheMultiPartEmail("", "", "", 
   "Failure: SoapUI Regression Test", "At least one of your 
   test runs failed. Check detailed results.", 
   "recipient@blarg.net")
   }
   else {
   context.email = new ApacheMultiPartEmail("", "", "", 
   "Pass: SoapUI Regression Test", 
   "None of your test runs failed.", 
   "recipient@blarg.net")
   }

* Here’s my email class definition, which closely resembles the example in the Apache Commons online docs. It was written to send emails via an Exchange SMTP server. Note that Apache Commons also offers a SimpleEmail class that would have worked just as well for this limited purpose.

import org.apache.commons.*;
public class ApacheMultiPartEmail {
public ApacheMultiPartEmail(attPath, attDescription, attName, msgSubject, msgMessage, msgRecipient) {
if (attPath != '' && attDescription != '' && attName != '') {
EmailAttachment attachment = new EmailAttachment();
attachment.setPath(attPath);
attachment.setDisposition(EmailAttachment.ATTACHMENT);
attachment.setDescription(attDescription);
attachment.setName(attName);
email.attach(attachment);
}

Email email = new MultiPartEmail();
email.setHostName("smtp.yourhostname.org"); // I hardcode this
email.setSmtpPort(yoursmtpport); // also hardcoded in the class definition
email.setFrom("desiredSenderEmailAddress"); //I hardcode this value in the class always to send the message from me. You could pass it in as a parameter too. 
email.setSubject(msgSubject);
email.setMsg(msgMessage);
email.addTo(msgRecipient);

email.send();
}
}