Nitin has posted 391 posts at DZone. View Full User Profile

Effective UI Testing with FlexMonkey - An Interview with Stu Stern

  • submit to reddit

In this interview, recorded at Adobe MAX 2009, Gorilla Logic CEO Stuart Stern talks to DZone about FlexMonkey, an Adobe AIR application used for testing Flex- and AIR-based applications. FlexMonkey also generates ActionScript-based testing scripts that you can easily include within a continuous integration environment.

The complete transcript of the interview has been provided below. 

DZone:  Can you tell us a little bit about some of the work you are doing at Gorilla Logic?

Stuart:  Sure. We founded Gorilla Logic in 2002. Prior to that, I actually ran Java Consulting worldwide for Sun for many years. Gorilla Logic started as a Java application development firm. We have since moved quite heavily into Flex as well as iPhone development, driven by the fact that, like many Java guys, we found Java to be a great platform but not such a great UI development platform anymore, so we've gotten quite heavily into Flex and out of that has come FlexMonkey.

DZone:  What is FlexMonkey?

Stuart:  FlexMonkey is a record and playback tool for Flex applications. You start it up, you point it to an SWF file, your application starts. And then as you interact with your application, it's recording your mouse clicks, text entry, all the interactions with the applications. It's recording those and you can then edit those and then play them back. And you can interleaf within those record events, assertions to verify the state of your applications at various points in the scripts. You can say, "After they click on the button and type in this field, these values should be in these fields, these colors should be set and so on and so forth." And then, you can replay those tests interactively or include them as part of a continuous integration environment.

DZone:  What are some of the developer‑centric features in FlexMonkey?

Stuart:  We were very much thinking about developers when we built the tool, so it was important to us that it be forgiving. Because as a developer, in QA testing, the code is coming in much more stable chunks. A developer is obviously changing the code constantly. So as you start building the app, there's going to be things that are going to appear hour‑by‑hour, that user interface is going to be changing as you are working on it. So we wanted you to be able to create very surgical tests that are going to be robust as you're changing the app. So as you move things around the screen, you're not going to break the test.

And we wanted it to be able to run from ANT, and then by association, from continuous integration frameworks such as Hudson and CruiseControl. And it does indeed do all those things, as well as actually generates code, so that you can actually add additional ActionScript to your test to test virtually any aspect of your application, examine internal states of things or even make changes to the underlying state of the application in code.

So in addition to just exercising the app through mouse clicks and typing and so on, you can also mix in ActionScript code, do other things to the application.

DZone:  Is FlexMonkey is primarily positioned as a unit testing framework, or does it also support integration and functional testing?

Stuart:  You can absolutely use it to do QA testing. We have been very surprised at the adoption we are getting in QA groups, because you can use it totally visually without any programming at all. It can really be used for all of the what some people are unit testing, functional testing, integration testing, you can use it in all of those scenarios. Because you use it to, as I mentioned before, surgically test very specific aspects of the app. But you can also ‑ because it is driving the user interface ‑ you could be running it with your backend stubbed out. You could also be running with a live backend and test all the way to the end.

You could even use it ‑ and we have used it ‑ to load test an application by adding action script to a generated test that we, you know, will loop the test hundreds of thousands of times. In fact, run multiple instances across many client machines and load test all the way end‑to‑end.

DZone:  How does FlexMonkey actually monitor my SWF file? Do I have to make (potentially invasive) changes to my  actionscript code?

Stuart:  Not at all. You never have to make any source code changes to use it. In fact, you can use it with most applications without even changing the SWF file, without changing the binary code at all. You start FlexMonkey, you tell it where your SWF file ‑ and that SWF file can be local or can in fact be a URL pointing to the server where it should be loaded ‑ and it dynamically loads. And then it has built itself, Flexmoneky, on top of the Flex automation framework and so actually hooks into the underlying Flex event system and that's how it's listening in on everything that's going on in your application.

DZone:  Can I employ FlexMonkey as part of a larger test-driven development strategy?

Stuart:  It depends how you define it. Certainly for test‑first development, certainly not. Because to use it effectively, in theory you can actually use FlexMonkey without recording, where you could just sit down and start writing the code. So from that perspective, in theory you could do test‑first development where you could just write this test and then start building the app. From a practical perspective, most people will build the part of the user interface that they want to record. As I mentioned, they might enhance the test with ActionScript, additional stuff that's beyond recording.

But it's definitely much easier to bootstrap your test by recording some interactions. Typically you're not writing your test first, you're doing what we call CALTAL, code a little, test a little kind of development.

It lends itself for people who aren't religious about whether you're writing the test and writing the code, which is really a whole design philosophy in terms of just though having testing being an integral part of the development process and test‑driven development. From that perspective, it absolutely works beautifully in that type of scenario.

DZone:  How has uptake been for the the project throughout the Flex community?

Stuart:  We're actually quite pleased with the adoption. In just the two months that it's been out in 1.0, it's existed ‑‑ there were early betas all the way going back a year, so it's been 12 months incubating its way to 1.0. The 1.0 release alone has now more than 2,500 registered users, and there are even more people that have download the source and are running their own bids. But the people that are downloading our binary, there are 2,500 people, and we get downloads every single day.

Talking to some of the Flex evangelists, we're hearing that it's just becoming quite popular. It's really going viral in the Flex community, although obviously given that there are hundreds of thousands of Flex developers out there, there's quite a lot of penetration still that we could see.

But there's nothing else that does what it does other than very expensive commercial tools, thousands of dollars a seat. You can't beat the price. It is free and open source. We expect actually that it's really going to be quite ubiquitous quite soon.

Going forward, there's a lot more enhancement. It's very feature‑complete right now in terms of being able to do rigorous user interface testing. It works with virtually any app. It'll work with Flex apps as well as Air apps, browser‑based apps.

As I say, there's really nothing. There are no features that it's screaming to have in order to be usable. But like any product, there are certainly enhancements that would be very helpful. We're going to add more features to allow it to be used by non‑programmers. You can work with the tool purely interactively to create tests, including creating assertions as part of your test.

You can visually mouse over components, pop up an inspector that shows you properties, and say this property should have this value at this point in the script, do that all visually without writing any code. But you can also then generate code that you can then go modify to add additional types of control logic, for example, or more complex types of verifications.

We want to put some stuff into the tool to allow more of those things to be done purely interactively without having to write any code. At the same time, we do not want to create a full‑blown testing‑scripting language, because you can already do that with ActionScript.

If you want to script it in code, you can just generate code out of the tool. In fact, as I said before, just start writing ActionScript using our API and do the test that way.

But certainly in the future releases of the tool, there will be more and more that you can do visually.

DZone:  Where can I download FlexMonkey?

Stuart: There are links to the source there. There's binary distribution you can install. All documentation is there. It's very well documented. There are tutorial videos. It's very easy to get up and running. The learning curve is like an hour.

DZone:  Stu, on behalf of the community, thank you very much for your time today. .

Stuart:  Thank you. Thanks for having me.

Published at DZone with permission of its author, Nitin Bharti.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)


Stan Berka replied on Thu, 2010/01/21 - 7:54pm

Yes, FlexMonkey is great. It allows to nicely test Flex application UI, *BUT* only if it supports Flex 4. We're starting a new project and we decided to go with the Flex 4 and Flash Builder 4. Will we be able to use FlexMonkey for functional tests in a month or two? That is the question.


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.