Gatling Version 2.1.7

Gatling Version 2.1.7

Gatling only have JSON, CSV, JDBC and some other support. So I wrote the XML support myself.

1 Setup the Gatling Project
plugins.sbt will enable all the plugins
addSbtPlugin("io.gatling" % "gatling-sbt" % "2.1.7")
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.6.0")

build.properties SBT version is update to date
sbt.version=0.13.8

.sbtopts to make the Memory Ok for Large Testing Data File
-J-Xmx1G
-J-XX:+UseConcMarkSweepGC
-J-XX:+CMSClassUnloadingEnabled
-J-Xss2M -Duser.timezone=GMT

Copy and use the default application.con, gatling.conf, logback.xml.

My base class to Prepare the Perf Env in PerformanceTesterEnvironment.scala
package com.sillycat.performance.base
import com.typesafe.config.ConfigFactory
trait PerformanceTesterEnvironment {

val config = ConfigFactory.load

val env = "env." + config.getString("build.env")

private def path(s: String*) = s.mkString(".")

def envStr(s: String): String = {
path(env, s)
}

val rampSeconds = 1

val rampNumOfUsers = config.getInt(envStr("ramp.user.num"))

val requestsPerUser = config.getInt(envStr("requests.per.user"))

val classifierHost = config.getString(envStr("classifier.host"))

}

A typical GET JSON API Simulation, ClassifierGetIndustrySimulation.scala
package com.sillycat.performance.classifier

import com.sillycat.performance.base.PerformanceTesterEnvironment
import com.sillycat.performance.feeder.CustomerFeederSupport

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._

class ClassifierGetIndustrySimulation extends Simulation with PerformanceTesterEnvironment with CustomerFeederSupport{

val httpConf = http
.baseURL(classifierHost)
.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8") // Here are the common headers
.doNotTrackHeader("1")
.acceptLanguageHeader("en-US,en;q=0.5")
.acceptEncodingHeader("gzip, deflate")
.userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0")

val headers =
Map("Accept" -> "application/json, text/javascript, */*; q=0.01",
"Keep-Alive" -> "180",
"X-Requested-With" -> "XMLHttpRequest",
"Content-type" -> "application/json")

//val jobTitleDescJSONFeed = jsonFile("job.title.desc.json").random
val jobTitleDescXMLFeed = xmlFile("job.title.desc.xml.small").random

val classifyJob2Industry = exec(http("classifyJob2Industry")
.get("/industry")
.headers(headers)
.queryParamMap(Map("title"->"${title}","description" -> "${description}", "format" -> "json"))
.check(status.is(200))
.check(bodyString.exists))

val multiScn = scenario("Classify Industry")
.repeat(requestsPerUser){ feed(jobTitleDescXMLFeed).exec(classifyJob2Industry) }

setUp(
multiScn.inject(rampUsers(rampNumOfUsers) over (1 seconds))
).protocols(httpConf)

}

2 Add Support to XML for Gatling
package com.sillycat.performance.feeder

import io.gatling.core.config.Resource
import io.gatling.core.feeder._
import io.gatling.core.validation.{Failure, Success, Validation}

trait CustomerFeederSupport {

def xmlFile(fileName: String): RecordSeqFeederBuilder[Any] = xmlFile(Resource.feeder(fileName))

def xmlFile(resource: Validation[Resource]): RecordSeqFeederBuilder[Any] =
feederBuilder(resource)(XMLFeederFileParser.parse)

def feederBuilder[T](resource: Validation[Resource])(recordParser: Resource => IndexedSeq[Record[T]]): RecordSeqFeederBuilder[T] =
resource match {
case Success(res) => RecordSeqFeederBuilder(recordParser(res))
case Failure(message) => throw new IllegalArgumentException(s"Could not locate feeder file; $message")
}

}

package com.sillycat.performance.feeder

import java.io.InputStream
import java.net.URL

import io.gatling.core.feeder.Record

import scala.collection._
import io.gatling.core.config.Resource
import io.gatling.core.util.IO._

import scala.xml.{NodeSeq, XML}

object XMLFeederFileParser {

def parse(resource: Resource): IndexedSeq[Record[Any]] =
withCloseable(resource.inputStream) { is =>
stream(is).toVector
}

def url(url: String): IndexedSeq[Record[Any]] =
withCloseable(new URL(url).openStream) { is =>
stream(is).toVector
}

def stream(is: InputStream): Iterator[Record[Any]] = {

val xml = scala.xml.XML.load(is)

(xml \ "JOB").iterator.collect {
case job:NodeSeq => {
Map("title" -> (job \\ "TITLE").text, "description" -> (job \\ "DESC").text).toMap
}
case _ => {
throw new IllegalArgumentException("Root element of XML feeder file isn't an array")
}
}

}

}

Tips
1 Codes to find the mismatched Tag in XML
package com.sillycat.performance

import scala.io.Source

/**
* Created by carl on 12/2/15.
*/
object MainTest extends App{

println("start calculating...")

var line_num = 0
var flag = 0
for(line <- Source.fromFile("/Users/carl/company/code/api-performance/user-files/data/classify.industry.xml.large").getLines()){
line_num = line_num + 1
if(line.contains("<JOB ")){
flag = flag + 1
}
if(line.contains("</JOB>")){
//println(line)
flag = flag - 1
}
if(flag > 1){
println(line_num + " " + flag)
flag = 0
}
}
}

2 XML Validation Error
Exception:
org.xml.sax.SAXParseException; lineNumber: 136589; columnNumber: 10; The content of elements must consist of well-formed character data or markup.
Exception
org.xml.sax.SAXParseException; lineNumber: 263; columnNumber: 49; The entity name must immediately follow the '&' in the entity reference.

Solution:
sublime - [Goto] - [Goto Line]
https://support.microsoft.com/en-us/kb/308060

Find the line number, lookup the special characters mapping, change < to < or > to >

3 Unicode Validation Error
Exception
org.xml.sax.SAXParseException; lineNumber: 1051001; columnNumber: 532; An invalid XML character (Unicode: 0x19) was found in the element content of the document.

Solution:
Right now, I just delete that.

4 HTTP Length Issue
Exception
org.jboss.netty.handler.codec.frame.TooLongFrameException: An HTTP line is larger than 4096 bytes.
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.readLine(HttpMessageDecoder.java:670) ~[io.netty.netty-3.10.4.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:184) ~[io.netty.netty-3.10.4.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:102) ~[io.netty.netty-3.10.4.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) ~[io.netty.netty-3.10.4.Final.jar:na]

Solution:
We should use POST via form data instead for large length of params.

References:
http://gatling.io/docs/2.1.6/extensions/sbt_plugin.html
https://github.com/gatling/gatling-sbt-plugin-demo