Extracting data from a LoadRunner results DB

I collegue wanted to record the resource usage from the weekly performance test automatically into an excel spreadsheet.  So this is how you do it. From excel choose Data -> Import External -> New Data Query and select a MS Access database from the dialogue box. Remember this is the access database created from running and saving the analysis, the default output.mdb produced at the end of a load test if for errors. Next you will need to open the access DB that was created when you saved the results analysis. After you have done this you will be able to use MS query to create the query.

You will need to join the Host, Event_map and Monitor_meter tables to construct the query. The equiry used is shown below, where it provides the resource average for an 1 hour of the test after the first 10 minutes.

SELECT Host.`Host Name`, Event_map.`Event Name`, Avg(Monitor_meter.Aminimum) AS ‘Avg ‘
FROM `C:\….\filename`.Event_map Event_map, `C:\….\filename`.Host Host, `C:\….\filename`.Monitor_meter Monitor_meter
WHERE Host.`Host ID` = Monitor_meter.`Host ID` AND Event_map.`Event ID` = Monitor_meter.`Event ID` AND ((Monitor_meter.`End Time`>=600 And Monitor_meter.`End Time`<=4200))
GROUP BY Host.`Host Name`, Event_map.`Event Name`

Once this is working in the query editor you can return back to excel and the data will be added to the spreadsheet

Python Packet Inspector for Network Captures

I recently was involved on a performance trouble shooting exercise for a company that uses Citrix to access their core ERP application. As part of the exercise individual users has taken packet captures using WireShark for key business transactions. The transactions types where the same for all the users but the users where are different locations. The common metric from the captures was the amount of data exchanged for the transaction and the session length. As there where many locations, transaction types and users the best way to do this was to automate the analysis.

I decided that I would use python as I was fairly familiar with the language and there where several libraries for WireShark analysis.

The chosen library was http://kiminewt.github.io/pyshark/ and a good tutorial on how to use this can be found here http://thepacketgeek.com/pyshark-using-the-packet-object/

The code imports, iterates through the files in the given directory (which should point to the wireshark capture files). When it finds a capture all the citrix packets are loaded into a capture array using a filter using the citrix port number
cap = pyshark.FileCapture(pcap_file,display_filter="tcp.port == 2598")

Once the cap array is populated then a loop iterates through the array summing the size of the payload data

#Iterate through the cap array
for i in cap:
# If the packet has payload data add that to the size counter
try:
if i.data:
size = size + int(i.data.len)
except:
pass

Finally, once the array has been processed then the payload size and session time is printed out. The session time is the timestamp of the last frame in the array which is held in the iteration variable i minus the timestamp for the first frame in the array. I have used epoch time so the result is in seconds. print file, ": is :",size,": size and :",float(i.frame_info.time_epoch)-float(cap[0].frame_info.time_epoch)

The complete code with a bit of error processing is here:


import pyshark
import os
directory = "/home/andrew/Documents/CaptureDirectory"

#Iterate through every file in the directory
for file in os.listdir(directory):
#Analyse if it a wireshark capture files
if file.endswith(".pcapng"):
#Populate cap array with packets matching using the citrix port 2598
size = 0
pcap_file = (directory + "/" + file)
cap = pyshark.FileCapture(pcap_file,display_filter="tcp.port == 2598")

#Iterate through the capute array
for i in cap:
# If the packet has payload data add that to the size counter
try:
if i.data:
size = size + int(i.data.len)
except:
pass
# Print out payload size and session duration
if size > 0:
print file, ": is :",size,": size and :",float(i.frame_info.time_epoch)-float(cap[0].frame_info.time_epoch)
else:
print file," No Citrix"

An example of the output is

andrew@debian:~$ python summary.py
smith-tran12.pcapng : is : 89636 : size and : 47.6105160713
davies-tran1.pcapng : is : 267292 : size and : 62.6023669243
smith-tran11.pcapng : is : 242545 : size and : 37.8602318764
kirby-tran1.pcapng No Citrix

Using R to detect growth in perfmon resource metrics

I have started using the statistical package R to detect any trends in performance test data. In this example I am looking to detect windows perfmon metrics that increase over the duration of a performance test.

You can install R an open source statistical analysis package from here http://www.r-project.org/

The code below can be cut and pasted into the R gui command line but you will have to change the first line of the script to use the directory holding the data file (procs.csv).

The comments should give you an idea what it is doing..

setwd("C:\\Documents and Settings\\alee\\My Documents\\Projects\\youProject\\")

# Load the Data isetednto a Data Frame

pData <- read.csv("procs.csv",sep=",",header=TRUE)

NumOfCols <- length(names(pData))

# Create a metrix to hold the gradients

slope <- 1:NumOfCols

# Loop through the metrics and calculate the slope

for(i in 2:NumOfCols ) {
     x <- 1:length(pData[[i]])
     y <- pData[[i]]
     # Ignore any blank columns
     if ( is.na(y[[1]]) )
          {slope[[i]] <- NA }
     else
     {
          fit <- lm( y ~ x)
          slope[[i]] <- fit[[1]][[2]]
     }
}

results=data.frame(metric=names(pData),Co=slope,OrgOrder=1:NumOfCols)

OrderResults <- results[order(-results$Co),]

#plot top 5 growing metrics

for(i in 1:5)
{
     OrderResults[i,3]
     plot(pData[[OrderResults[i,3]]],type="o",col="blue")
     title(main=names(pData[OrderResults[i,3]]), col.main="black", font.main=4)
     # Prompt for Enter so plot stays on screen long enough to be read
     readline(prompt = "Pause. Press to continue...")
}