sorcket java_Karate：让Web服务测试变得简单

The 0.9.0 release is a big one. Karate no longer uses Cucumber behind the scenes and the feature-file parser and execution-engine were re-written from scratch. 100% backwards compatibility has been maintained, all your existing tests will run fine - and even IDE support has been preserved.

This took some serious engineering effort, but was totally worth it ! We can now move fast and improve the engine without worrying about the evolution of Cucumber versions. For example, the parallel-runner now executes even Scenario-s within a Feature in parallel, and that includes data-driven Examples within Scenario Outline-s.

Which leads us to the only possible breaking change you may encounter - Scenarios will be executed in parallel by default. See below.

Breaking Change

There are absolutely no breaking changes for syntax and Java API, and it is highly likely your existing Karate tests would run as-is without any changes. But in case you have Scenario-s that depend on each-other (which obviously is a bad-practice and absolutely NOT recommended) you will have to use the @parallel=false tag to force them to run in sequence - when using the parallel runner. Just place the tag above the Feature level in your test-script so that it applies to all Scenario-s within that Feature. Read more.

While upgrading, use this opportunity to ensure that your tests are indeed independent and can run in any order !

TestNG Support Deprecated

The documentation has always warned that JUnit is strongly recommended, and we have been announcing this deprecation in advance. This is not going to be an issue because even in the rare chance that you have gone "all in" on TestNG, the Maven surefire plugin supports JUnit and TestNG in the same project. Note that JUnit 5 support has been introduced. Keep in mind that the parallel runner (which is what you will be using most, e.g. for CI) is pure-Java and it does not matter if the runtime is JUnit or TestNG.

Planned Deprecations

The next release will deprecate the @CucumberOptions annotation and the CucumberRunner class, please plan to switch to com.intuit.karate.KarateOptions and com.intuit.karate.Runner. And com.intuit.karate.Results will eventually replace KarateStats. All the examples and documentation have been updated. So this is the recommended pattern:

Results results = Runner.parallel(getClass(), 5);

assertTrue(results.getErrorMessages(), results.getFailCount() == 0);

In fact for tests in dev mode (not the parallel runner), if you start using JUnit 5 you may not need the @KarateOptions (previously @CucumberOptions) annotation any more.

On the command line, instead of cucumber.options - you can start using karate.options.

To be clear, we will only turn on deprecation warnings in the next release, the old stuff will still work in the next release (1.0) and will stop working (as per plan) only later, in the release after 1.0 - so in short, you don't need to change anything for now.

karate-netty now part of karate-core

Your existing projects are not affected, but now you no longer need to use the karate-netty artifact if you want to use mocks or test-doubles, all of that goodness is in the main JAR itself. In other words, when you use karate-apache, you get the test-doubles capabilities as well.

New and Notable

Parallel Scenarios

Already mentioned above, and there is a new HTML report that allows you to visualize the parallel efficiency: see video.

JUnit 5 support

Thanks to the JUnit team and @sormuras for his awesome support and PR-s ! You can have multiple methods in the same test-class which should reduce a lot of clutter in dev-mode. And the API is elegant, nice and DRY. Read more.

Re-vamped Karate UI

Great for demos and de-bugging tests, a big improvement is that you can even step through call-ed features. You can import Postman collections like before, but that capability is still considered experimental. Please do contribute if you can ! See video. Read more.

Thanks to @babusekaran for some PR-s !

Lower Case and Headers

As per the HTTP spec header names should be case-insensitive. You can force all header key-values to be lower-case:

*configure lowerCaseResponseHeaders = true

We also made it easy to take any JSON and brute-force all of it to lower-case, which can be useful in some situations:

*def response = karate.lowerCase(response)

responseBytes and requestBytes

An additional built-in variable responseBytes will hold the response as a byte-array, useful when dealing with binary-content. And the match syntax even works with byte-arrays, no there's no need to do a custom array comparison any more:

Andmatch responseBytes == read('test.pdf')

Similarly, for the test-doubles / karate-netty side, requestBytes holds the incoming request byte-array.

Call Tag Selector

You can now choose a single Scenario when using the call (or callonce) keyword. Just append it to the feature name like so:

call read('classpath:my-signin.feature@sometagname')

Gatling

The above call tag selector concept allows you to better compose load tests out of a large suite of existing Karate tests. Two other big improvements are the ability to customize the "report name" of any HTTP request, and to even test any Java code, not just HTTP ! As an example, see the Karate-gRPC project. Read more.

Report Verbosity

Hide steps in the report when configured or show only steps that don't begin with the * prefix. Read more

WebSocket and Async

There are certainly not many testing frameworks that support websockets. And with just two API methods: karate.signal(result) and karate.listen(timeout) - you can easily set up tests to wait for async events such as message-queues.

Read more

Retry Support

No more messing around with JavaScript loops and an additional file to call, now provide a condition before the method step and Karate will do the rest. Read more.

Dynamic Scenario Outlines

Examples no longer have to have a fixed number of rows at run-time. Use JSON arrays, that can even be dynamic and created at run-time - but retain the same read-ability and report-friendliness of the Scenario Outline. Read more.

CSV File Support

Karate can now read CSV files out of the box - and as you can imagine this goes really well with the Dynamic Scenario Outlines described above. So if you read() a file with a *.csv extension, Karate will convert it into a JSON array, and a header row is always expected. Read more.

Image Embedding

You can now use karate.embed(bytes, mimeType) to embed any kind of content into the test output and these would be viewable in the HTML report. Read more.

Chrome Browser Automation

You may have heard of Puppeteer and "Headless Chrome" - and guess what, Karate has first-class support for Headless Chrome now ! This sets us up for UI automation (see the next section) but - Karate is probably one of the few libraries out there that gives you the capability to save HTML from a URL you choose - into PNG or even PDF. Read more.

UI Automation (experimental)

Yes, why not - provided API testing is feature-complete and mature. We strongly feel that if we get this right, and actually solve the problem of "flaky tests", Karate would be a serious force in the world of test-automation. In our opinion, there are quite a few things that give Karate an advantage:

clean code base, LOTs of tests, one-click git-clone and maven build dev-experience, easy for contributors

battle-tested parallel execution, that can be easily extended into a "grid" solution in future, see video

UI to debug and step-through tests, that we can easily extend into a record-replay "IDE" solution in future, see video

cross-browser support from day-one, see video

deep webdriver protocol trace-ability, see video

Karate's regression tests for UI automation use the Karate test-doubles to serve HTML (true "dog-fooding" :), which gives us a unique advantage, for e.g. we can simulate situations like slow-loading pages with ease

and we think one of the keys to stable and speedy UI test-automation is to move test-doubles higher in the "test pyramid" - for example, think of testing only a chunk of your UI (not the whole end-to-end) after perhaps injecting cookies and mock HTTP end-points

since we have Gatling support, we are in a unique position to apply it in creative ways to UI testing in the future

Karate's unique approach to JavaScript embedding may be just what the doctor ordered for taming the HTML DOM and highly dynamic pages, AJAX and all

And Karate's assertions for JSON and XML are built-in, no extra library required

support for Websocket that solves for Chrome (the same approach as Puppeteer) and which can be applied for tighter control over non-Chrome browsers in the future

Unified API that solves for web, desktop and mobile - aligned with the WebDriver W3C standard, see video

The UI automation capability is deemed experimental as of now and the API can change. We are releasing this so that you can experiment and perhaps be inspired to contribute because of all the above reasons ! Read more.

Pull Requests

Issue Register