Difference between revisions of "ESciDoc Release Lifecycle"

From MPDLMediaWiki
Jump to navigation Jump to search
 
(69 intermediate revisions by 2 users not shown)
Line 37: Line 37:
== Perform a release ==
== Perform a release ==


Before you want to make a release be sure to have the appropriate rights for uploading to the nexus server. Without these rights, a release can not be deployed and fails!
Before you start with a release build make sure to have the appropriate rights for uploading to the nexus server. Without these rights, a release can not be deployed and fails!
Currently there's no way to check the user roles in nexus by a standard user. So you have to ask a nexus administrator, if you have the appropriate rights to deploy releases.
Currently there's no way to check the user roles in nexus by a standard user. So you have to ask a nexus administrator, if you have the appropriate rights to deploy releases.


After the rights are validated, you have to decide whether you want to do a release of the complete project or just a single module release.  
After the rights are validated, you have to decide whether you want to do a release of the complete project or just a single module release.


=== Project release ===
== Project release ==


A project release means you want to release everything in the project, every submodule and the parent ''pom.xml'' (this is the ''pom.xml'' in the root folder of the project). Even submodules  
A project release means you want to release everything in the project, every submodule and the parent ''pom.xml'' (this is the ''pom.xml'' in the root folder of the project). Even submodules  
Line 52: Line 52:
release form, you have to take care of the correct inter-project module dependencies by yourself.
release form, you have to take care of the correct inter-project module dependencies by yourself.


For the release itself, there are two possibilities. A graphically assisted release run. Then you would choose Continuum. Or if you prefer a release on the local workstation, do a command line release.
It makes sense to checkout a sources to a special release build directory in order to avoid an interception with the local source developement instance.


It makes sense to checkout a sources to special release directory in order to avoid an interception with the local svn instance.
==== Release with Continuum ====
''At the moment Continuum has some problems with a maven plugin affecting the release mechanism, so the graphical release is not available right now.''
==== Command line release on local workstation ====
==== Command line release on local workstation ====


* Preparation for scp to work
* Preparation for scp to work


: You need an account with sufficient rights on the machine the website will be deployed (=target machine). Make sure the target machine is contained in the known_host file of your local workstation. This can be achieved by log in successfully via ssh.
: You need an account with sufficient rights on the machine the website will be deployed (=target machine)to. The account on your local workstation you are running the release build also needs to have rights for overwriting former releases on the target machine. Make sure the target machine is contained in the ''known_host'' file of your local workstation. This can be achieved by log in successfully via ssh.


: On your local workstation generate an rsa key without passphrase using ssh_keygen. Export the public key to the authorized_keys of the target machine. See [http://moveitsupport.ipswitch.com/moveit/doc/en/MOVEitDMZ_SSH_SpecificClients_OpenSSHWindows.htm ssh]. Location of the ssh files on both machines is ~/..ssh
: On your local workstation generate an rsa key without passphrase using ''ssh_keygen''. Export the public key to the ''authorized_keys'' file of the target machine. See [http://moveitsupport.ipswitch.com/moveit/doc/en/MOVEitDMZ_SSH_SpecificClients_OpenSSHWindows.htm ssh]. Location of the ssh files on both machines is ~/..ssh.


: Check if there's the following section in your ~/.m2/settings.xml
: Check if there's the following section in your ~/.m2/settings.xml
Line 72: Line 68:
     <id>website</id>
     <id>website</id>
     <!-- ssh username -->
     <!-- ssh username -->
     <username></username>
     <username>myName</username>
     <passPhrase></passPhrase>
     <privateKey>/path/to/private/key</privateKey>
    <filePermissions>666</filePermissions>
     <passphrase></passphrase>
     <directoryPermissions>777</directoryPermissions>
   </server>
   </server>
   ...
   ...
</pre>
</pre>


* Changes
* Update and commit pubman online download page ([https://github.com/MPDL/PubMan/tree/master/pubman/src/site/xdoc/get_pubman/download.xml git], [http://pubman.mpdl.mpg.de/projects/pubman/get_pubman/download.html target url]) with the release specific comments and download links, if needed.
: The changes properties define the behavior of the announcement plugin.  
: This plugin is responsible to send an email to a mail address if a new release is done. It is also responsible to download all the jira bugs and present them on the project webpage. That's why, you have to provide
: your jira account. ;)
: The private key to upload the documentation is in the ''/home/jboss'' folder. Copy the file ''id_rsa'' file to your local machine and set the path to this file in the ''settings.xml''.


: Next you have to edit the changes.xml file for each module you want to release. This file resides in '''src/changes/changes.xml''' in the root folder. You might see something like this:
* Make sure you have the proper profiles activated in your ''settings.xml''. For the available profiles see http://colab.mpdl.mpg.de/mediawiki/Maven_settings_and_profiles
The standard profiles to activate for a release build are
<pre>
<pre>
<document>
dev, integration-tests, deploy-artifacts, env-pure, standalone, env-release-build, mysettings
  <properties>
    <title>Common services project</title>
    <author email="endres@mpdl.mpg.de">Thomas Endres</author>
  </properties>
  <body>
    <release version="1.0.0" date="2008-02-21" description="Initial release">
      <action dev="tendres" type="update">
        Lots of changes for the maven build environment.
      </action>
    </release>
    <release version="1.0.1" date="2008-03-07" description="Release of all modules">
    </release>
    <release version="1.0.2-SNAPSHOT" date="2008-03-11" description="current development">
    <action dev="wfrank" type="update">
        Applied changes required for new framework release(version 0.9.1 build 224)
      </action>
    </release>
  </body>
</document>
</pre>
</pre>


: As you can see there's a section with a snapshot version number. These are the recently finished development actions on the module. These actions haven't been released yet. Also remember that there are still modules
* Create a new release directory on your local machine and checkout the sources concerned into this directory.  
: which doesn't have this snapshot section yet. If it misses, please add it. ;)
: To release the above module would now involve the following steps:
**  First change the snapshot version
<pre>
<pre>
...
git clone <git url aus copy url...>
<release version="1.0.2-SNAPSHOT" date="2008-03-11" description="current development">
cd ...
...
git checkout <branchName> (z.B. master, 1.3-wildfly)
 
</pre>
</pre>
: to a standard version
 
Go to the project root folder, if you want to release the whole project; otherwise to the modules root folder. Now type
<pre>
<pre>
...
mvn release:prepare -DpreparationGoals="clean install" -DupdateReleaseInfo=true
<release version="1.0.2" date="2008-04-07" description="Single module release">
...
</pre>
</pre>
** Then open a new snapshot section for further development actions
:Maven proposes a release version for every module concerned (e.g if the last snapshot version is 1.1.5-SNAPSHOT, the default release version proposed is 1.1.5). If a module has been modified, increase the version according the above section. Related modules (e.g cone, cone_presentation, cone_ear) should get the same version. The sources get tagged with a label containing the release version. This information is stored locally in the ''release.properties'' file in the directory where the release preparation has been called. Having finished the preparation successfully maven increases the versions in the ''pom.xml'' files for the next developement version (e.g if the release version is 1.1.5 the following developement version will be 1.1.6-SNAPHOT)and commits the modified ''pom.xml'' to subversion.
:For testing purposes first start the maven build prepare process with -DdryRun=true. Then no update to the SCM is done.
:If you want to allow using SNAPSHOT dependencies use -DignoreSnapshots=true.
 
* To discard a release, enter
<pre>
<pre>
<release version="1.0.3-SNAPSHOT" date="2008-04-07" description="Current development">
mvn release:rollback
    </release>
</pre>
</pre>
** Checkin the modified file
: All project POMs are reverted back to their pre-release state locally, and also in the SCM if the previous release command was able to successfully make changes in the SCM to the POMs. Precondition is that the previous release descriptor release.properties is still available in the local working copy. You will need to manually remove the tag from your SCM:
 
* Update and commit pubman online download page ([https://zim02.gwdg.de/repos/pubman/trunk/pubman/src/site/xdoc/get_pubman/download.xml svn], [http://pubman.mpdl.mpg.de/projects/pubman/get_pubman/download.html target url]) with the release specific comments and download links, if needed.
 
* Create a new release directory on your local machine and checkout the sources concerned into this directory. Go to the project root folder, if you want to release the whole project; otherwise to the modules root folder). Now type
<pre>
<pre>
mvn release:prepare -DpreparationGoals="clean install"
git tag -d <tag>
git push --tags -f
</pre>
</pre>
:Maven proposes a release version for every module concerned. If a module has been modified, increase the version according the above section. Related modules (e.g cone, cone_presentation, cone_ear) should get the same version. The sources get tagged with a label containing the release version. This information is stored locally in the ''release.properties'' file in the directory where the release preparation has been called. Having finished the preparation successfully maven increases the versions in the ''pom.xml'' files for the next developement version (e.g if the release version is 1.1.5 the following developement version will be 1.1.6-SNAPHOT).


* If you want to discard the release, enter
* If the prepare was successful, the release can be performed. First make sure that you have sufficient rights for writing on the target machine by scp.
<pre>
<pre>
mvn release:rollback
cd srv/www/staging/projects
chown -R user:group *
</pre>
</pre>
: This will remove the new tag, and rollback the version changes in the ''pom.xml' files.
: where ''user:group'' are user and group used for scp. Now enter
 
* If the prepare was successful, the release can be performed. Do this by typing:
<pre>
<pre>
mvn release:perform
mvn release:perform
</pre>  
</pre>  
: The system now creates all the needed artifacts (jars, javadoc, etc.) for the release and starts the tests. If these were okay, the artifacts will be transfered to [http://rd.mpdl.mpg.de/nexus/content/groups/public/ Nexus].  
: A new repository is created contained in the release repository on nexus as target for the pom.xml and site information. The name of the new repository contains the tag generated by the maven release prepare build. The system now creates all the needed artifacts (jars, javadoc, etc.) for the release and starts the tests. If these were okay, the artifacts will be transfered to [http://rd.mpdl.mpg.de/nexus/content/groups/public/ Nexus]. For gaining speed direct the output to an text file.  
* To publish the snapshot version, you should now do a
 
<pre>
mvn clean deploy
</pre>
: This makes the new snapshot version available, so that no developer should have some 'unable to download artifact' errors. ;)


Further information on the release mechanism can be found on the Homepage of the [http://maven.apache.org/plugins/maven-release-plugin/ Maven release plugin].
Further information on the release mechanism can be found on the Homepage of the [http://maven.apache.org/plugins/maven-release-plugin/ Maven release plugin].


=== Further steps to do after a release ===
== Further steps to do after a release ==
* Update the source tree in your development environment. Check the version numbers and the versions of the dependencies in the ''pom.xml'' files to be increased to the new developement version (should end with -SNAPSHOT).
* Copy over the released ear to the qa-machine for testing
* Copy over the released ear to the qa-machine for testing
* Write an email to escidoc-dev to inform that a new release has been made
* Clean up the old SNAPSHOT ear on the dev-machine
* Clean up the old SNAPSHOT ear on the dev-machine
* Open a new release in JIRA and close the open release, which you just have released
* Open a new release in JIRA and close the open release, which you just have released


== Publish a release ==
== Publish a release ==
* Copy documentation from ''escidoc1.escidoc.mpg.de:/srv/www/staging/projects'' to ''/escidoc1.escidoc.mpg.de:/srv/www/00_default/projects''
* Copy documentation to ''pubman.mpdl.mpg.de:/srv/www/projects/pubman/get_pubman''
* Copy ear to the appropriate live instance and check the configuration
* Copy also ear and installer to this destination


== Deal with errors ==
== Hints - Deal with errors ==
* If a release fails and the system states that the tag you want to create is already in the svn repository, you have to delete it manually directly in the repository:
* For testing purposes first start the maven build process with ''-DdryRun = true''. For detailed description see http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html
** Open an svn browsing tools like 'kdesvn' for Linux or the 'repo-browser' (right click in SVN Tortoise) for Windows
* If a release fails and the system states that the tag you want to create is already in the svn repository, you have to delete it manually directly in the repository
: --[[User:Makarenko|Makarenko]] 16:22, 3 March 2010 (UTC): In kdesvn: File->Open->URL to the repo. (e.g. <code>https://subversion.mpdl.mpg.de/repos/pubman</code>)
* If the maven release:perform fails remove the pom.xml and site information in the corresponding release directory on nexus before calling maven release:perform for a second time. Otherwise the upload on nexus will fail.
** Simply delete the tag you want to create newly
** Be careful to not delete the wrong tag!!!!!


* If you get a strange svn error just do a
* Maven-release-plugin 2.0 throws java.lang.StringIndexOutOfBoundsException during mvn release:pepare of the PubMan projects. Upgrade to Maven-release-plugin 2.1 resolved the problem. Also see http://stackoverflow.com/questions/5517408/maven2-string-index-out-of-range-1-on-getcommonbasedir-during-rewrite-poms-for
<pre>
svn up
</pre>


[[Category:ESciDoc|Release Lifecycle]]
[[Category:ESciDoc|Release Lifecycle]]
[[Category:How To]]
[[Category:How To]]
[[Category:Release Log]]
[[Category:Release Log]]

Latest revision as of 09:59, 18 April 2018

Phases of a release[edit]

  • Tag the code in the SCM with a version name
  • Deploy the current documentation to the webserver
  • Deploy the build artifacts to the nexus server
  • Modify the version number in the poms

Identification of a release[edit]

The identification of a release is called version number. In the eSciDoc project these version numbers consist of three decimal numbers separated by dots. If a new release is made, one of these numbers will be incremented.

If the version is a candidate version between two releases, a SNAPSHOT identifier will be appended at the end. So e.g 2.0.18-SNAPSHOT means, the next release version will be 2.0.18. The table below indicates the dependency between software changes and this identification.


X. X. X. -SNAPSHOT

- Milestone changes

- Interface changes

- Greater design changes

- Smaller design changes

- Bugfixes

this indicates it's an intermediate snapshot version


Now if the new version identifier is specified, a release can be performed.

Perform a release[edit]

Before you start with a release build make sure to have the appropriate rights for uploading to the nexus server. Without these rights, a release can not be deployed and fails! Currently there's no way to check the user roles in nexus by a standard user. So you have to ask a nexus administrator, if you have the appropriate rights to deploy releases.

After the rights are validated, you have to decide whether you want to do a release of the complete project or just a single module release.

Project release[edit]

A project release means you want to release everything in the project, every submodule and the parent pom.xml (this is the pom.xml in the root folder of the project). Even submodules which haven't changed are being released. This procedure makes sense, if a lot of submodules have changed and you don't want to release every module one-by-one. If the parent pom.xml has changed you always have to do a project release, since every submodule changes its dependency.

Single module release[edit]

A single module release means you want to release just a single module of a project. This is normally the case, if there are changes just in one or two submodules of a project. Remember that in this release form, you have to take care of the correct inter-project module dependencies by yourself.

It makes sense to checkout a sources to a special release build directory in order to avoid an interception with the local source developement instance.

Command line release on local workstation[edit]

  • Preparation for scp to work
You need an account with sufficient rights on the machine the website will be deployed (=target machine)to. The account on your local workstation you are running the release build also needs to have rights for overwriting former releases on the target machine. Make sure the target machine is contained in the known_host file of your local workstation. This can be achieved by log in successfully via ssh.
On your local workstation generate an rsa key without passphrase using ssh_keygen. Export the public key to the authorized_keys file of the target machine. See ssh. Location of the ssh files on both machines is ~/..ssh.
Check if there's the following section in your ~/.m2/settings.xml
  ...
  <server>
    <id>website</id>
    <!-- ssh username -->
    <username>myName</username>
    <privateKey>/path/to/private/key</privateKey>
    <passphrase></passphrase>
  </server>
  ...
  • Update and commit pubman online download page (git, target url) with the release specific comments and download links, if needed.

The standard profiles to activate for a release build are

dev, integration-tests, deploy-artifacts, env-pure, standalone, env-release-build, mysettings
  • Create a new release directory on your local machine and checkout the sources concerned into this directory.
git clone <git url aus copy url...>
cd ...
git checkout <branchName> (z.B. master, 1.3-wildfly)

Go to the project root folder, if you want to release the whole project; otherwise to the modules root folder. Now type

mvn release:prepare -DpreparationGoals="clean install" -DupdateReleaseInfo=true
Maven proposes a release version for every module concerned (e.g if the last snapshot version is 1.1.5-SNAPSHOT, the default release version proposed is 1.1.5). If a module has been modified, increase the version according the above section. Related modules (e.g cone, cone_presentation, cone_ear) should get the same version. The sources get tagged with a label containing the release version. This information is stored locally in the release.properties file in the directory where the release preparation has been called. Having finished the preparation successfully maven increases the versions in the pom.xml files for the next developement version (e.g if the release version is 1.1.5 the following developement version will be 1.1.6-SNAPHOT)and commits the modified pom.xml to subversion.
For testing purposes first start the maven build prepare process with -DdryRun=true. Then no update to the SCM is done.
If you want to allow using SNAPSHOT dependencies use -DignoreSnapshots=true.
  • To discard a release, enter
mvn release:rollback
All project POMs are reverted back to their pre-release state locally, and also in the SCM if the previous release command was able to successfully make changes in the SCM to the POMs. Precondition is that the previous release descriptor release.properties is still available in the local working copy. You will need to manually remove the tag from your SCM:
git tag -d <tag>
git push --tags -f
  • If the prepare was successful, the release can be performed. First make sure that you have sufficient rights for writing on the target machine by scp.
cd srv/www/staging/projects
chown -R user:group *
where user:group are user and group used for scp. Now enter
mvn release:perform
A new repository is created contained in the release repository on nexus as target for the pom.xml and site information. The name of the new repository contains the tag generated by the maven release prepare build. The system now creates all the needed artifacts (jars, javadoc, etc.) for the release and starts the tests. If these were okay, the artifacts will be transfered to Nexus. For gaining speed direct the output to an text file.


Further information on the release mechanism can be found on the Homepage of the Maven release plugin.

Further steps to do after a release[edit]

  • Copy over the released ear to the qa-machine for testing
  • Clean up the old SNAPSHOT ear on the dev-machine
  • Open a new release in JIRA and close the open release, which you just have released

Publish a release[edit]

  • Copy documentation to pubman.mpdl.mpg.de:/srv/www/projects/pubman/get_pubman
  • Copy also ear and installer to this destination

Hints - Deal with errors[edit]

  • For testing purposes first start the maven build process with -DdryRun = true. For detailed description see http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html
  • If a release fails and the system states that the tag you want to create is already in the svn repository, you have to delete it manually directly in the repository
  • If the maven release:perform fails remove the pom.xml and site information in the corresponding release directory on nexus before calling maven release:perform for a second time. Otherwise the upload on nexus will fail.