Automatic GUI tests with selenium

General

Selenium is a portable software testing framework for web applications. The tests can be written with a Firefox plugin. The plugin records all your steps and afterwards integrates it into a testcase. The tests can also be written in several programming languages.

The tests are run on your local workstation. That means, when have deployed your ear, a selenium testing instance is started. This instance is then used to run all the earlier defined testcases.

Selenium architecture

http://seleniumhq.org/docs/_images/chapt5_img01_Architecture_Diagram_Simple.png

Selenium Tool Suite

Selenium-IDE (Integrated Development Environment/firefox plugin)

• is intended as a rapid prototyping tool
• allows developing and running tests without the need for programming skills
• Selenium Commands (Selenese) features:
  • Element can be located with
    • Identifier
    • Id
    • Name
    • XPath
    • Hyperlinks (Link Text)
    • DOM
    • CSS
  • Text can be matched
    • Globbing
    • RegExp
    • Exact Patterns
  • AndWait Commands (for AJAX as well)
  • Stored Variables
  • Stored JS Snippet Evals
  • Pop-ups handling (Alerts, Confirmations, Multiply Windows)
• can only run it’s test scripts against Firefox (since it is FF plugin)
• programming in java is not needed
• doesn't support iteration or condition statements (like testing each element of a variable length list)
  • complete list what is not directly supported:
    • condition statements
    • iteration
    • logging and reporting of test results
    • error handling, particularly unexpected errors
    • database testing
    • test case grouping
    • re-execution of failed tests
    • test case dependency
    • screenshot capture of test failures

Selenium-RC (Remote Control/java)

• java programming
• covers all features of selenium
• supports iterations and condition statements
• tests of the any complexity
• integration with JUnit
• integration with maven

Selenium-WebDriver (new Selenium API)

• new direction of Selenium development (i.e. Selenium 2.0)
• has no limitations of Selenium-RC 1.0
• has backward compatibility with Selenium-RC 1.0

Selenium-Grid

• solution to scale for large test suites or test suites that must be run in multiple environments

Selenium-Core (HTML inerface TestRunner.html)

• deprecated

Flow Control (iterations and conditions) in Selenium

Possible options:

• Run the script using Selenium-RC and Java client library (java programming is needed)
• Run a small JavaScript snippet from within the script using the storeEval command
• Install the goto_sel_ide.js extension (for Selenium-IDE)

Firefox selenium plugin (Selenium-IDE) installation

• Point your Firefox browser to https://addons.mozilla.org/en-US/firefox/addon/2079 and install the selenium plugin
• After restarting the browser you should see 'Selenium IDE' in your 'Tools' folder.
• If you open it, you get a toolbox for creating, recording and verifying your testcases.

Possible PubMan testing workflows

Create/Playback of testcases/test suits with Selenium-IDE

• Prerequisites:
  • Selenium-IDE plugin
  • Local SVN working copy of PubMan testcases repo
  • Testcase specifications in JIRA and PubMan testing repo, xsl files
• Creation of new testcase/test suite
  1. Open PubMan page to be tested in Firefox
  2. Open Selenium-IDE and start recording (see here and here)
  3. Proceed PubMan test case according to specification in the Firefox
  4. Stop recording
  5. Modify Selenium Commands in Selenium-IDE if needed

     Comments and annotates can be inserted directly in testcase (See here)

  6. Save testcase in working copy under the appropriate name
  7. Create or update already created Test Suite.

     Note: Test case files should not have to be co-located with the test suite file that invokes them. And on Mac OS and Linux systems, that is indeed the case. However, a bug prevents Windows users from being able to place the test cases elsewhere than with the test suite that invokes them.

  8. Commit created testcase/test suit to the SVN repo https://subversion.mpdl.mpg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/
• Playback testcase/test suite
  1. Open Test Suite from working copy with Selenium-IDE->File->Open Test Suite
  2. Playback (See here )

Selenium Firefox UI

Further reading: Selenium-IDE documentation page.

Playback of testcases/test suits with Selenium-RC

• Prerequisites:
  • Local SVN working copy of PubMan testcases repo
  • Java =>1.5
  • Browsers Firefox [, IE, Safari, Opera, etc.]

java -jar selenium-server.jar -htmlSuite "*firefox" "http://www.google.com" "c:\absolute\path\to\my\HTMLSuite.html" "c:\absolute\path\to\my\results.html"

Playback of testcases/test suits with MAVEN

• Prerequisites:
  • Local SVN working copy of PubMan testcases repo
  • Local SVN working copy of pubman_integration
  • Java =>1.5
  • maven
  • Browsers Firefox [, IE, Safari, Opera, etc.]

Maven integration of selenium

TO BE UPDATED

Selenium is running a PubMan testcase.

To use selenium in a maven build environment, the selenium maven plugin is used. You can add this plugin simply by including the plugin into your pom.xml The plugin here is encapsulated in a profile, which means unless this profile is activated, no tests are made.

  <profile>
            <id>integration-test</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.cargo</groupId>
                        <artifactId>cargo-maven2-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>start-container</id>
                                <phase>pre-integration-test</phase>
                                <goals>
                                    <goal>start</goal>
                                </goals>
                            </execution>
                            <execution>
                                <id>stop-container</id>
                                <phase>post-integration-test</phase>
                                <goals>
                                    <goal>stop</goal>
                                </goals>
                            </execution>
                        </executions>
                        <configuration>
                            <wait>false</wait>
                            <container>
                                <containerId>jboss42x</containerId>
                                <zipUrlInstaller>
                                    <url>${cargo.jboss.url}</url>
                                    <installDir>${cargo.install.dir}</installDir>
                                </zipUrlInstaller>
                            </container>
                            <configuration>
                                <type>existing</type>
                                <home>${project.build.directory}/jboss/server</home>
                                <properties>
                                    <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                                    <cargo.rmi.port>${cargo.rmi.port}</cargo.rmi.port>
                                    <cargo.logging>${cargo.log}</cargo.logging>
                                    <cargo.jvmargs>${cargo.jvmargs}</cargo.jvmargs>
                                    <cargo.jboss.configuration>default</cargo.jboss.configuration>
                                </properties>
                            </configuration>
                        </configuration>
                    </plugin>
                    <plugin>
                        <artifactId>maven-antrun-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>launch-selenium</id>
                                <phase>integration-test</phase>
                                <configuration>
                                    <tasks>
                                        <taskdef resource="selenium-ant.properties">
                                            <classpath refid="maven.plugin.classpath" />
                                        </taskdef>
                                        <selenese suite="src/test/resources/selenium/TestSuite.html" browser="*firefox /usr/lib/firefox/firefox-2-bin" timeoutInSeconds="180"
                                            port="5555" results="${project.build.directory}/selenium-firefox-results.html" startURL="http://localhost:${cargo.port}/pubman/" />
                                    </tasks>
                                </configuration>
                                <goals>
                                    <goal>run</goal>
                                </goals>
                            </execution>
                        </executions>
                        <dependencies>
                            <dependency>
                                <groupId>ant</groupId>
                                <artifactId>ant-nodeps</artifactId>
                                <version>1.6.5</version>
                            </dependency>
                            <dependency>
                                <groupId>org.openqa.selenium.server</groupId>
                                <artifactId>selenium-server</artifactId>
                                <version>1.0-beta-1</version>
                            </dependency>
                        </dependencies>
                    </plugin>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>selenium-maven-plugin</artifactId>
                        <version>1.0-beta-1</version>
                        <executions>
                            <execution>
                                <id>start-selenium</id>
                                <phase>pre-integration-test</phase>
                                <goals>
                                    <goal>start-server</goal>
                                </goals>
                                <configuration>
                                    <background>true</background>
                                    <port>6666</port>
                                    <logOutput>true</logOutput>
                                    <multiWindow>true</multiWindow>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                    <plugin>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>surefire-it</id>
                                <phase>integration-test</phase>
                                <goals>
                                    <goal>test</goal>
                                </goals>
                                <configuration>
                                    <excludes>
                                        <exclude>none</exclude>
                                    </excludes>
                                    <includes>
                                        <include>**/selenium/*Test.java</include>
                                    </includes>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>

                </plugins>
            </build>
        </profile>

The profile defines the jboss instance, where, in this case PubMan, is started and it defines the selenium client instance. In this case the Firefox browser is used. To start the GUI tests for this PubMan example simply type:

mvn -Pintegration-test clean install

To be done in the PubMan projects

• Check the compatibility with Windows
• Finish use of static identifiers within dynamic pages
• Integrate different browsers than Firefox
• Integrate Java GUI testcases into Maven

Selenium (introduction, architecture)

Media:TestTool_Selenium.pdf

PubMan and Selenium tests

Media:Pubman_and_Selenium_tests.ppt

Related Links

• Selenium docs
• JIRA tesing cases
• PubMan test cases SVN repo
• pubman_ear selenium JUnits

Enterprise Archive File Formate

Integrated Development Environment

Expression Language Used to Navigate and to Find Structures in XML Documents

Document Object Model

Cascading Style Sheets

Rights Checking

Application Programming Interface

Global Release Identifier

Hypertext Markup Language

Publication Management

Apache Subversion

Operating System

Internet Explorer

Graphical User Interface