Running Flex functional tests using Hudson, Tomcat and RIA Test

In the Flex world its proven difficult and time consuming to test UI using unit test based approaches. Functional tests, based on Flex automation, seem to provide a better alternative. In my previous post I evaluated using RIAtest for this purpose.

In this post I go over integrating RIATest with Hudson so our tests run after every build and provide benefits of Continuous Integration.

Setup

  • Hudson build server (Windows XP)
  • RIATest installed on the build server
  • Tomcat installed as a service, on the build server

Problems to solve

  1. Integrate RIATest with Hudson: I leverage the fact that RIATest can be run from the command line. Hudson allows you to add a post build step which runs a .bat/.sh script.
  2. Making Hudson aware of build success/failure: RIATest command line execution returns an exit code.  Non 0 in case of test failure or other problems. Correspondingly, Hudson fails the build if the return code of the post build script is non-zero.
  3. Running the tests: One of the problems was to fire up the web app in an automated manner, before the tests can be run. For us this involves deploying the app on a JEE servlet container.  My solution was to install and run Tomcat as a Windows service. This can be started and stopped from the build script.

Hudson cool tip

Since the script is non-trivial it makes sense to check in into source control. Using a bit of Hudson coolness, we can get Hudson to run the script which was checked out as a part of the workspace.


Script

REM Stop Tomcat (if its running)
SC query Tomcat6 | FIND "STATE" | FIND "STOPPED"
if ERRORLEVEL 1 (
 sc stop Tomcat6
 sleep 30
)

REM Clean webapps
del /F /S /Q %CATALINA_HOME%\logs\*.*
if EXIST %CATALINA_HOME%\webapps\webapp.war (del /F /S /Q %CATALINA_HOME%\webapps\webapp.war)

REM Deploy war
copy xxx\xxx\webapp.war %CATALINA_HOME%\webapps 

REM Start Tomcat
sc start Tomcat6
sleep 120

REM Run RIATest
if NOT EXIST %TEST_RESULTS_DIR% (mkdir %TEST_RESULTS_DIR%)

riatest --run --batch 
--project="xxx\xxx\src\functional\riatest\MyTests.rtp" 
--output-xml="%TEST_RESULTS_DIR%\riatest-results.xml" 
--output="%TEST_RESULTS_DIR%\riatest-results.txt" 
--run-scripts="-//;+//XX//testScenarios"

if ERRORLEVEL 127 goto UNKNOWN_ERROR
if ERRORLEVEL 9 goto ERROR_LIMIT
if ERRORLEVEL 8 goto TIME_LIMIT
if ERRORLEVEL 7 goto AGENT_DISCONNECT
if ERRORLEVEL 6 goto TIMEOUT_WAIT_AGENT
if ERRORLEVEL 5 goto SCRIPT_RUN_ERROR
if ERRORLEVEL 4 goto CANT_OPEN_PROJECT
if ERRORLEVEL 3 goto CANT_OPEN_OUTPUT
if ERRORLEVEL 2 goto INVALID_PARAMS
if ERRORLEVEL 1 goto NOTHING_TO_DO
if ERRORLEVEL 0 goto OK

:UNKNOWN_ERROR
set TEST_STATUS=127
goto PROCESS_RESULTS

:ERROR_LIMIT
set TEST_STATUS=9
goto PROCESS_RESULTS

:TIME_LIMIT
set TEST_STATUS=8
goto PROCESS_RESULTS

:AGENT_DISCONNECT
set TEST_STATUS=7
goto PROCESS_RESULTS

:TIMEOUT_WAIT_AGENT
set TEST_STATUS=6
goto PROCESS_RESULTS

:SCRIPT_RUN_ERROR
set TEST_STATUS=5
goto PROCESS_RESULTS

:CANT_OPEN_PROJECT
set TEST_STATUS=4
goto PROCESS_RESULTS

:CANT_OPEN_OUTPUT
set TEST_STATUS=3
goto PROCESS_RESULTS

:INVALID_PARAMS
set TEST_STATUS=2
goto PROCESS_RESULTS

:NOTHING_TO_DO
set TEST_STATUS=1
goto PROCESS_RESULTS

:OK
set TEST_STATUS=0
goto PROCESS_RESULTS

: PROCESS_RESULTS
type %TEST_RESULTS_DIR%\riatest-results.txt

REM Write HTML test report
rtxml2html /i %TEST_RESULTS_DIR%\riatest-results.xml /o %TEST_RESULTS_DIR%\ria-test-results.html

REM Stop Tomcat
sc stop Tomcat6
sleep 30

REM Clean up webapps
del /F /S /Q %CATALINA_HOME%\logs\*.*
del /F /S /Q %CATALINA_HOME%\webapps\webapp.war

REM Hudson looks for this exit code
exit /b %TEST_STATUS%

if ERRORLEVEL
Advertisements
This entry was posted in Tech bits. Bookmark the permalink.

One Response to Running Flex functional tests using Hudson, Tomcat and RIA Test

  1. Nekobasu says:

    This is great stuff. A shell script for Mac OS X to do the same would be great. If you run Cygwin on Windows, you can use the same shell script for both Windows and Mac (albeit with conditional branching for path names).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s