Difference between revisions of "Automatic GUI tests with selenium"

From MPDLMediaWiki
Jump to navigation Jump to search
m
 
Line 82: Line 82:
* Prerequisites:
* Prerequisites:
** [[#Firefox_selenium_plugin_.28Selenium-IDE.29_installation|Selenium-IDE plugin]]
** [[#Firefox_selenium_plugin_.28Selenium-IDE.29_installation|Selenium-IDE plugin]]
** Local SVN working copy of [https://zim02.gwdg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/ PubMan testcases repo]
** Local SVN working copy of [https://subversion.mpdl.mpg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/ PubMan testcases repo]
** Testcase  specifications in [http://jira.mpdl.mpg.de/secure/IssueNavigator.jspa?mode=hide&requestId=10210 JIRA] and [https://zim02.gwdg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests PubMan testing repo, xsl files]
** Testcase  specifications in [http://jira.mpdl.mpg.de/secure/IssueNavigator.jspa?mode=hide&requestId=10210 JIRA] and [https://subversion.mpdl.mpg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests PubMan testing repo, xsl files]
* Creation of new testcase/test suite
* Creation of new testcase/test suite
*# Open PubMan page to be tested in Firefox
*# Open PubMan page to be tested in Firefox
Line 94: Line 94:
*# Create or update already created [http://seleniumhq.org/docs/03_selenium_ide.html#writing-a-test-suite Test Suite].  
*# Create or update already created [http://seleniumhq.org/docs/03_selenium_ide.html#writing-a-test-suite Test Suite].  
*#: ''Note:'' Test case files should not have to be co-located with the test suite file that invokes them. And on Mac OS and Linux systems, that is indeed the case. However, a bug prevents Windows users from being able to place the test cases elsewhere than with the test suite that invokes them.
*#: ''Note:'' Test case files should not have to be co-located with the test suite file that invokes them. And on Mac OS and Linux systems, that is indeed the case. However, a bug prevents Windows users from being able to place the test cases elsewhere than with the test suite that invokes them.
*# Commit created testcase/test suit to the SVN repo https://zim02.gwdg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/
*# Commit created testcase/test suit to the SVN repo https://subversion.mpdl.mpg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/
* Playback testcase/test suite
* Playback testcase/test suite
*# Open Test Suite from working copy with Selenium-IDE->File->Open Test Suite  
*# Open Test Suite from working copy with Selenium-IDE->File->Open Test Suite  
Line 105: Line 105:
=== Playback of testcases/test suits with Selenium-RC ===
=== Playback of testcases/test suits with Selenium-RC ===
* Prerequisites:
* Prerequisites:
** Local SVN working copy of [https://zim02.gwdg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/ PubMan testcases repo]
** Local SVN working copy of [https://subversion.mpdl.mpg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/ PubMan testcases repo]
** Java =>1.5
** Java =>1.5
** Browsers Firefox [, IE, Safari, Opera, etc.]  
** Browsers Firefox [, IE, Safari, Opera, etc.]  
Line 113: Line 113:
=== Playback of testcases/test suits with MAVEN  ===
=== Playback of testcases/test suits with MAVEN  ===
* Prerequisites:
* Prerequisites:
** Local SVN working copy of [https://zim02.gwdg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/ PubMan testcases repo]
** Local SVN working copy of [https://subversion.mpdl.mpg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/ PubMan testcases repo]
** Local SVN working copy of pubman_integration
** Local SVN working copy of pubman_integration
** Java =>1.5
** Java =>1.5
Line 274: Line 274:
* Selenium [http://seleniumhq.org/docs/ docs]
* Selenium [http://seleniumhq.org/docs/ docs]
* [http://jira.mpdl.mpg.de/secure/IssueNavigator.jspa?mode=hide&requestId=10210 JIRA tesing cases]
* [http://jira.mpdl.mpg.de/secure/IssueNavigator.jspa?mode=hide&requestId=10210 JIRA tesing cases]
* PubMan test cases [https://zim02.gwdg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/ SVN repo]
* PubMan test cases [https://subversion.mpdl.mpg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/ SVN repo]
* [https://zim02.gwdg.de/repos/pubman/trunk/pubman/puman_ear/src/test/java/de/mpg/de/escidoc/pubman/test/gui pubman_ear selenium JUnits]  
* [https://subversion.mpdl.mpg.de/repos/pubman/trunk/pubman/puman_ear/src/test/java/de/mpg/de/escidoc/pubman/test/gui pubman_ear selenium JUnits]  
   
   



Latest revision as of 15:49, 25 April 2012

General[edit]

Selenium is a portable software testing framework for web applications. The tests can be written with a Firefox plugin. The plugin records all your steps and afterwards integrates it into a testcase. The tests can also be written in several programming languages.

The tests are run on your local workstation. That means, when have deployed your ear, a selenium testing instance is started. This instance is then used to run all the earlier defined testcases.

Selenium architecture[edit]

http://seleniumhq.org/docs/_images/chapt5_img01_Architecture_Diagram_Simple.png

Selenium Tool Suite[edit]

Selenium-IDE (Integrated Development Environment/firefox plugin)[edit]

  • is intended as a rapid prototyping tool
  • allows developing and running tests without the need for programming skills
  • Selenium Commands (Selenese) features:
    • Element can be located with
      • Identifier
      • Id
      • Name
      • XPath
      • Hyperlinks (Link Text)
      • DOM
      • CSS
    • Text can be matched
      • Globbing
      • RegExp
      • Exact Patterns
    • AndWait Commands (for AJAX as well)
    • Stored Variables
    • Stored JS Snippet Evals
    • Pop-ups handling (Alerts, Confirmations, Multiply Windows)
  • can only run it’s test scripts against Firefox (since it is FF plugin)
  • programming in java is not needed
  • doesn't support iteration or condition statements (like testing each element of a variable length list)
    • complete list what is not directly supported:
      • condition statements
      • iteration
      • logging and reporting of test results
      • error handling, particularly unexpected errors
      • database testing
      • test case grouping
      • re-execution of failed tests
      • test case dependency
      • screenshot capture of test failures

Selenium-RC (Remote Control/java)[edit]

  • java programming
  • covers all features of selenium
  • supports iterations and condition statements
  • tests of the any complexity
  • integration with JUnit
  • integration with maven

Selenium-WebDriver (new Selenium API)[edit]

  • new direction of Selenium development (i.e. Selenium 2.0)
  • has no limitations of Selenium-RC 1.0
  • has backward compatibility with Selenium-RC 1.0

Selenium-Grid[edit]

  • solution to scale for large test suites or test suites that must be run in multiple environments

Selenium-Core (HTML inerface TestRunner.html)[edit]

  • deprecated

Flow Control (iterations and conditions) in Selenium[edit]

Possible options:

  • Run the script using Selenium-RC and Java client library (java programming is needed)
  • Run a small JavaScript snippet from within the script using the storeEval command
  • Install the goto_sel_ide.js extension (for Selenium-IDE)

Firefox selenium plugin (Selenium-IDE) installation[edit]

  • Point your Firefox browser to https://addons.mozilla.org/en-US/firefox/addon/2079 and install the selenium plugin
  • After restarting the browser you should see 'Selenium IDE' in your 'Tools' folder.
  • If you open it, you get a toolbox for creating, recording and verifying your testcases.

Possible PubMan testing workflows[edit]

Create/Playback of testcases/test suits with Selenium-IDE[edit]

  • Prerequisites:
  • Creation of new testcase/test suite
    1. Open PubMan page to be tested in Firefox
    2. Open Selenium-IDE and start recording (see here and here)
    3. Proceed PubMan test case according to specification in the Firefox
    4. Stop recording
    5. Modify Selenium Commands in Selenium-IDE if needed
      Comments and annotates can be inserted directly in testcase (See here)
    6. Save testcase in working copy under the appropriate name
    7. Create or update already created Test Suite.
      Note: Test case files should not have to be co-located with the test suite file that invokes them. And on Mac OS and Linux systems, that is indeed the case. However, a bug prevents Windows users from being able to place the test cases elsewhere than with the test suite that invokes them.
    8. Commit created testcase/test suit to the SVN repo https://subversion.mpdl.mpg.de/repos/testing/trunk/SeleniumTests/Pubman_Selenium_Tests/
  • Playback testcase/test suite
    1. Open Test Suite from working copy with Selenium-IDE->File->Open Test Suite
    2. Playback (See here )
Error creating thumbnail: /bin/bash: /usr/bin/convert: No such file or directory Error code: 127
Selenium Firefox UI

Further reading: Selenium-IDE documentation page.

Playback of testcases/test suits with Selenium-RC[edit]

  • Prerequisites:
java -jar selenium-server.jar -htmlSuite "*firefox" "http://www.google.com" "c:\absolute\path\to\my\HTMLSuite.html" "c:\absolute\path\to\my\results.html"

Playback of testcases/test suits with MAVEN[edit]

  • Prerequisites:
    • Local SVN working copy of PubMan testcases repo
    • Local SVN working copy of pubman_integration
    • Java =>1.5
    • maven
    • Browsers Firefox [, IE, Safari, Opera, etc.]

Maven integration of selenium[edit]

TO BE UPDATED

Error creating thumbnail: /bin/bash: /usr/bin/convert: No such file or directory Error code: 127
Selenium is running a PubMan testcase.

To use selenium in a maven build environment, the selenium maven plugin is used. You can add this plugin simply by including the plugin into your pom.xml The plugin here is encapsulated in a profile, which means unless this profile is activated, no tests are made.

  <profile>
            <id>integration-test</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.cargo</groupId>
                        <artifactId>cargo-maven2-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>start-container</id>
                                <phase>pre-integration-test</phase>
                                <goals>
                                    <goal>start</goal>
                                </goals>
                            </execution>
                            <execution>
                                <id>stop-container</id>
                                <phase>post-integration-test</phase>
                                <goals>
                                    <goal>stop</goal>
                                </goals>
                            </execution>
                        </executions>
                        <configuration>
                            <wait>false</wait>
                            <container>
                                <containerId>jboss42x</containerId>
                                <zipUrlInstaller>
                                    <url>${cargo.jboss.url}</url>
                                    <installDir>${cargo.install.dir}</installDir>
                                </zipUrlInstaller>
                            </container>
                            <configuration>
                                <type>existing</type>
                                <home>${project.build.directory}/jboss/server</home>
                                <properties>
                                    <cargo.servlet.port>${cargo.port}</cargo.servlet.port>
                                    <cargo.rmi.port>${cargo.rmi.port}</cargo.rmi.port>
                                    <cargo.logging>${cargo.log}</cargo.logging>
                                    <cargo.jvmargs>${cargo.jvmargs}</cargo.jvmargs>
                                    <cargo.jboss.configuration>default</cargo.jboss.configuration>
                                </properties>
                            </configuration>
                        </configuration>
                    </plugin>
                    <plugin>
                        <artifactId>maven-antrun-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>launch-selenium</id>
                                <phase>integration-test</phase>
                                <configuration>
                                    <tasks>
                                        <taskdef resource="selenium-ant.properties">
                                            <classpath refid="maven.plugin.classpath" />
                                        </taskdef>
                                        <selenese suite="src/test/resources/selenium/TestSuite.html" browser="*firefox /usr/lib/firefox/firefox-2-bin" timeoutInSeconds="180"
                                            port="5555" results="${project.build.directory}/selenium-firefox-results.html" startURL="http://localhost:${cargo.port}/pubman/" />
                                    </tasks>
                                </configuration>
                                <goals>
                                    <goal>run</goal>
                                </goals>
                            </execution>
                        </executions>
                        <dependencies>
                            <dependency>
                                <groupId>ant</groupId>
                                <artifactId>ant-nodeps</artifactId>
                                <version>1.6.5</version>
                            </dependency>
                            <dependency>
                                <groupId>org.openqa.selenium.server</groupId>
                                <artifactId>selenium-server</artifactId>
                                <version>1.0-beta-1</version>
                            </dependency>
                        </dependencies>
                    </plugin>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>selenium-maven-plugin</artifactId>
                        <version>1.0-beta-1</version>
                        <executions>
                            <execution>
                                <id>start-selenium</id>
                                <phase>pre-integration-test</phase>
                                <goals>
                                    <goal>start-server</goal>
                                </goals>
                                <configuration>
                                    <background>true</background>
                                    <port>6666</port>
                                    <logOutput>true</logOutput>
                                    <multiWindow>true</multiWindow>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                    <plugin>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>surefire-it</id>
                                <phase>integration-test</phase>
                                <goals>
                                    <goal>test</goal>
                                </goals>
                                <configuration>
                                    <excludes>
                                        <exclude>none</exclude>
                                    </excludes>
                                    <includes>
                                        <include>**/selenium/*Test.java</include>
                                    </includes>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>

                </plugins>
            </build>
        </profile>

The profile defines the jboss instance, where, in this case PubMan, is started and it defines the selenium client instance. In this case the Firefox browser is used. To start the GUI tests for this PubMan example simply type:

mvn -Pintegration-test clean install

To be done in the PubMan projects[edit]

  • Check the compatibility with Windows
  • Finish use of static identifiers within dynamic pages
  • Integrate different browsers than Firefox
  • Integrate Java GUI testcases into Maven

Selenium (introduction, architecture)[edit]

Media:TestTool_Selenium.pdf


PubMan and Selenium tests[edit]

Media:Pubman_and_Selenium_tests.ppt

Related Links[edit]