Difference between revisions of "ESciDoc Release Lifecycle"

From MPDLMediaWiki
Jump to navigation Jump to search
 
(54 intermediate revisions by the same user not shown)
Line 37: Line 37:
== Perform a release ==
== Perform a release ==


Before you want to make a release be sure to have the appropriate rights for uploading to the nexus server. Without these rights, a release can not be deployed and fails!
Before you start with a release build make sure to have the appropriate rights for uploading to the nexus server. Without these rights, a release can not be deployed and fails!
Currently there's no way to check the user roles in nexus by a standard user. So you have to ask a nexus administrator, if you have the appropriate rights to deploy releases.
Currently there's no way to check the user roles in nexus by a standard user. So you have to ask a nexus administrator, if you have the appropriate rights to deploy releases.


After the rights are validated, you have to decide whether you want to do a release of the complete project or just a single module release.  
After the rights are validated, you have to decide whether you want to do a release of the complete project or just a single module release.


=== Project release ===
== Project release ==


A project release means you want to release everything in the project, every submodule and the parent ''pom.xml'' (this is the ''pom.xml'' in the root folder of the project). Even submodules  
A project release means you want to release everything in the project, every submodule and the parent ''pom.xml'' (this is the ''pom.xml'' in the root folder of the project). Even submodules  
Line 52: Line 52:
release form, you have to take care of the correct inter-project module dependencies by yourself.
release form, you have to take care of the correct inter-project module dependencies by yourself.


For the release itself, there are two possibilities. A graphically assisted release run. Then you would choose Continuum. Or if you prefer a release on the local workstation, do a command line release.
It makes sense to checkout a sources to a special release build directory in order to avoid an interception with the local source developement instance.


It makes sense to checkout a sources to special release directory in order to avoid an interception with the local svn instance.
==== Release with Continuum ====
''At the moment Continuum has some problems with a maven plugin affecting the release mechanism, so the graphical release is not available right now.''
==== Command line release on local workstation ====
==== Command line release on local workstation ====


* Preparation for scp to work
* Preparation for scp to work


: You need an account with sufficient rights on the machine the website will be deployed (=target machine). Make sure the target machine is contained in the ''known_host'' file of your local workstation. This can be achieved by log in successfully via ssh.
: You need an account with sufficient rights on the machine the website will be deployed (=target machine)to. The account on your local workstation you are running the release build also needs to have rights for overwriting former releases on the target machine. Make sure the target machine is contained in the ''known_host'' file of your local workstation. This can be achieved by log in successfully via ssh.


: On your local workstation generate an rsa key without passphrase using ''ssh_keygen''. Export the public key to the ''authorized_keys'' file of the target machine. See [http://moveitsupport.ipswitch.com/moveit/doc/en/MOVEitDMZ_SSH_SpecificClients_OpenSSHWindows.htm ssh]. Location of the ssh files on both machines is ~/..ssh.
: On your local workstation generate an rsa key without passphrase using ''ssh_keygen''. Export the public key to the ''authorized_keys'' file of the target machine. See [http://moveitsupport.ipswitch.com/moveit/doc/en/MOVEitDMZ_SSH_SpecificClients_OpenSSHWindows.htm ssh]. Location of the ssh files on both machines is ~/..ssh.
Line 72: Line 68:
     <id>website</id>
     <id>website</id>
     <!-- ssh username -->
     <!-- ssh username -->
     <username></username>
     <username>myName</username>
     <passPhrase></passPhrase>
     <privateKey>/path/to/private/key</privateKey>
    <filePermissions>666</filePermissions>
     <passphrase></passphrase>
     <directoryPermissions>777</directoryPermissions>
   </server>
   </server>
   ...
   ...
</pre>
</pre>


* Update and commit pubman online download page ([https://subversion.mpdl.mpg.de/repos/pubman/trunk/pubman/src/site/xdoc/get_pubman/download.xml svn], [http://pubman.mpdl.mpg.de/projects/pubman/get_pubman/download.html target url]) with the release specific comments and download links, if needed.
* Update and commit pubman online download page ([https://github.com/MPDL/PubMan/tree/master/pubman/src/site/xdoc/get_pubman/download.xml git], [http://pubman.mpdl.mpg.de/projects/pubman/get_pubman/download.html target url]) with the release specific comments and download links, if needed.
 
* Make sure you have the proper profiles activated in your ''settings.xml''. For the available profiles see http://colab.mpdl.mpg.de/mediawiki/Maven_settings_and_profiles
The standard profiles to activate for a release build are
<pre>
dev, integration-tests, deploy-artifacts, env-pure, standalone, env-release-build, mysettings
</pre>
 
* Create a new release directory on your local machine and checkout the sources concerned into this directory.
<pre>
git clone <git url aus copy url...>
cd ...
git checkout <branchName> (z.B. master, 1.3-wildfly)


* Create a new release directory on your local machine and checkout the sources concerned into this directory. Go to the project root folder, if you want to release the whole project; otherwise to the modules root folder). Now type
</pre>
 
Go to the project root folder, if you want to release the whole project; otherwise to the modules root folder. Now type
<pre>
<pre>
mvn release:prepare -DpreparationGoals="clean install" -DupdateReleaseInfo=true
mvn release:prepare -DpreparationGoals="clean install" -DupdateReleaseInfo=true
</pre>
</pre>
:Maven proposes a release version for every module concerned (e.g if the last snapshot version is 1.1.5-SNAPSHOT, the default release version proposed is 1.1.5). If a module has been modified, increase the version according the above section. Related modules (e.g cone, cone_presentation, cone_ear) should get the same version. The sources get tagged with a label containing the release version. This information is stored locally in the ''release.properties'' file in the directory where the release preparation has been called. Having finished the preparation successfully maven increases the versions in the ''pom.xml'' files for the next developement version (e.g if the release version is 1.1.5 the following developement version will be 1.1.6-SNAPHOT)and commits the modified ''pom.xml'' to subversion.
:Maven proposes a release version for every module concerned (e.g if the last snapshot version is 1.1.5-SNAPSHOT, the default release version proposed is 1.1.5). If a module has been modified, increase the version according the above section. Related modules (e.g cone, cone_presentation, cone_ear) should get the same version. The sources get tagged with a label containing the release version. This information is stored locally in the ''release.properties'' file in the directory where the release preparation has been called. Having finished the preparation successfully maven increases the versions in the ''pom.xml'' files for the next developement version (e.g if the release version is 1.1.5 the following developement version will be 1.1.6-SNAPHOT)and commits the modified ''pom.xml'' to subversion.
:For testing purposes first start the maven build prepare process with -DdryRun=true. Then no update to the SCM is done.
:If you want to allow using SNAPSHOT dependencies use -DignoreSnapshots=true.


* If you want to discard the release, enter
* To discard a release, enter
<pre>
<pre>
mvn release:rollback
mvn release:rollback
</pre>
</pre>
: This will remove the new tag, and rollback the version changes in the ''pom.xml' files.
: All project POMs are reverted back to their pre-release state locally, and also in the SCM if the previous release command was able to successfully make changes in the SCM to the POMs. Precondition is that the previous release descriptor release.properties is still available in the local working copy. You will need to manually remove the tag from your SCM:
<pre>
git tag -d <tag>
git push --tags -f
</pre>


* If the prepare was successful, the release can be performed. Do this by typing:
* If the prepare was successful, the release can be performed. First make sure that you have sufficient rights for writing on the target machine by scp.
<pre>
cd srv/www/staging/projects
chown -R user:group *
</pre>
: where ''user:group'' are user and group used for scp. Now enter
<pre>
<pre>
mvn release:perform
mvn release:perform
</pre>  
</pre>  
: A new repository is created contained in the release repository on nexus as target for the pom.xml and site information. The name of the new repository contains the tag generated by the maven release prepare build. The system now creates all the needed artifacts (jars, javadoc, etc.) for the release and starts the tests. If these were okay, the artifacts will be transfered to [http://rd.mpdl.mpg.de/nexus/content/groups/public/ Nexus].  
: A new repository is created contained in the release repository on nexus as target for the pom.xml and site information. The name of the new repository contains the tag generated by the maven release prepare build. The system now creates all the needed artifacts (jars, javadoc, etc.) for the release and starts the tests. If these were okay, the artifacts will be transfered to [http://rd.mpdl.mpg.de/nexus/content/groups/public/ Nexus]. For gaining speed direct the output to an text file.  
* To publish the snapshot version, you should now do a
 
<pre>
mvn clean deploy
</pre>
: This makes the new snapshot version available, so that no developer should have some 'unable to download artifact' errors. ;)


Further information on the release mechanism can be found on the Homepage of the [http://maven.apache.org/plugins/maven-release-plugin/ Maven release plugin].
Further information on the release mechanism can be found on the Homepage of the [http://maven.apache.org/plugins/maven-release-plugin/ Maven release plugin].


=== Further steps to do after a release ===
== Further steps to do after a release ==
* Update the source tree in your development environment. You get the new SNAPSHOT version in the ''pom.xml'' files committed by the maven release:prepare process. Check the version numbers and the versions of the dependencies in the ''pom.xml'' files. The version numbers of the dependencies in the ''pom.xml'' files are adapted automatically as long as the dependencies belong to the same release process; otherwise they have to be adapted manually. This means: the version numbers of dependencies belonging to e.g. common_services have to be adapted manually in the PubMan ''pom.xml'' files.
* Copy over the released ear to the qa-machine for testing
* Copy over the released ear to the qa-machine for testing
* Write an email to escidoc-dev to inform that a new release has been made
* Clean up the old SNAPSHOT ear on the dev-machine
* Clean up the old SNAPSHOT ear on the dev-machine
* Open a new release in JIRA and close the open release, which you just have released
* Open a new release in JIRA and close the open release, which you just have released


== Publish a release ==
== Publish a release ==
* Copy documentation from ''escidoc1.escidoc.mpg.de:/srv/www/staging/projects'' to ''/escidoc1.escidoc.mpg.de:/srv/www/00_default/projects''
* Copy documentation to ''pubman.mpdl.mpg.de:/srv/www/projects/pubman/get_pubman''
* Copy ear to the appropriate live instance and check the configuration
* Copy also ear and installer to this destination


== Hints - Deal with errors ==
== Hints - Deal with errors ==
* For testing purposes first start the maven build process with ''-DdryRun = true''. For detailed description see http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.htm
* For testing purposes first start the maven build process with ''-DdryRun = true''. For detailed description see http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html
* If a release fails and the system states that the tag you want to create is already in the svn repository, you have to delete it manually directly in the repository:
* If a release fails and the system states that the tag you want to create is already in the svn repository, you have to delete it manually directly in the repository
** Open an svn browsing tools like 'kdesvn' for Linux or the 'repo-browser' (right click in SVN Tortoise) for Windows
* If the maven release:perform fails remove the pom.xml and site information in the corresponding release directory on nexus before calling maven release:perform for a second time. Otherwise the upload on nexus will fail.
: --[[User:Makarenko|Makarenko]] 16:22, 3 March 2010 (UTC): In kdesvn: File->Open->URL to the repo. (e.g. <code>https://subversion.mpdl.mpg.de/repos/pubman</code>)
** Simply delete the tag you want to create newly
** Be careful to not delete the wrong tag!!!!!


* If you get a strange svn error just do a
* Maven-release-plugin 2.0 throws java.lang.StringIndexOutOfBoundsException during mvn release:pepare of the PubMan projects. Upgrade to Maven-release-plugin 2.1 resolved the problem. Also see http://stackoverflow.com/questions/5517408/maven2-string-index-out-of-range-1-on-getcommonbasedir-during-rewrite-poms-for
<pre>
svn up
</pre>
 
* If the maven release:perform fails remove the pom.xml and site information in the corresponding release directory on nexus before calling maven release:perform for a second time. Otherwise the upload on nexus will fail.


[[Category:ESciDoc|Release Lifecycle]]
[[Category:ESciDoc|Release Lifecycle]]
[[Category:How To]]
[[Category:How To]]
[[Category:Release Log]]
[[Category:Release Log]]

Latest revision as of 07:59, 18 April 2018

Phases of a release[edit]

  • Tag the code in the SCM with a version name
  • Deploy the current documentation to the webserver
  • Deploy the build artifacts to the nexus server
  • Modify the version number in the poms

Identification of a release[edit]

The identification of a release is called version number. In the eSciDoc project these version numbers consist of three decimal numbers separated by dots. If a new release is made, one of these numbers will be incremented.

If the version is a candidate version between two releases, a SNAPSHOT identifier will be appended at the end. So e.g 2.0.18-SNAPSHOT means, the next release version will be 2.0.18. The table below indicates the dependency between software changes and this identification.


X. X. X. -SNAPSHOT

- Milestone changes

- Interface changes

- Greater design changes

- Smaller design changes

- Bugfixes

this indicates it's an intermediate snapshot version


Now if the new version identifier is specified, a release can be performed.

Perform a release[edit]

Before you start with a release build make sure to have the appropriate rights for uploading to the nexus server. Without these rights, a release can not be deployed and fails! Currently there's no way to check the user roles in nexus by a standard user. So you have to ask a nexus administrator, if you have the appropriate rights to deploy releases.

After the rights are validated, you have to decide whether you want to do a release of the complete project or just a single module release.

Project release[edit]

A project release means you want to release everything in the project, every submodule and the parent pom.xml (this is the pom.xml in the root folder of the project). Even submodules which haven't changed are being released. This procedure makes sense, if a lot of submodules have changed and you don't want to release every module one-by-one. If the parent pom.xml has changed you always have to do a project release, since every submodule changes its dependency.

Single module release[edit]

A single module release means you want to release just a single module of a project. This is normally the case, if there are changes just in one or two submodules of a project. Remember that in this release form, you have to take care of the correct inter-project module dependencies by yourself.

It makes sense to checkout a sources to a special release build directory in order to avoid an interception with the local source developement instance.

Command line release on local workstation[edit]

  • Preparation for scp to work
You need an account with sufficient rights on the machine the website will be deployed (=target machine)to. The account on your local workstation you are running the release build also needs to have rights for overwriting former releases on the target machine. Make sure the target machine is contained in the known_host file of your local workstation. This can be achieved by log in successfully via ssh.
On your local workstation generate an rsa key without passphrase using ssh_keygen. Export the public key to the authorized_keys file of the target machine. See ssh. Location of the ssh files on both machines is ~/..ssh.
Check if there's the following section in your ~/.m2/settings.xml
  ...
  <server>
    <id>website</id>
    <!-- ssh username -->
    <username>myName</username>
    <privateKey>/path/to/private/key</privateKey>
    <passphrase></passphrase>
  </server>
  ...
  • Update and commit pubman online download page (git, target url) with the release specific comments and download links, if needed.

The standard profiles to activate for a release build are

dev, integration-tests, deploy-artifacts, env-pure, standalone, env-release-build, mysettings
  • Create a new release directory on your local machine and checkout the sources concerned into this directory.
git clone <git url aus copy url...>
cd ...
git checkout <branchName> (z.B. master, 1.3-wildfly)

Go to the project root folder, if you want to release the whole project; otherwise to the modules root folder. Now type

mvn release:prepare -DpreparationGoals="clean install" -DupdateReleaseInfo=true
Maven proposes a release version for every module concerned (e.g if the last snapshot version is 1.1.5-SNAPSHOT, the default release version proposed is 1.1.5). If a module has been modified, increase the version according the above section. Related modules (e.g cone, cone_presentation, cone_ear) should get the same version. The sources get tagged with a label containing the release version. This information is stored locally in the release.properties file in the directory where the release preparation has been called. Having finished the preparation successfully maven increases the versions in the pom.xml files for the next developement version (e.g if the release version is 1.1.5 the following developement version will be 1.1.6-SNAPHOT)and commits the modified pom.xml to subversion.
For testing purposes first start the maven build prepare process with -DdryRun=true. Then no update to the SCM is done.
If you want to allow using SNAPSHOT dependencies use -DignoreSnapshots=true.
  • To discard a release, enter
mvn release:rollback
All project POMs are reverted back to their pre-release state locally, and also in the SCM if the previous release command was able to successfully make changes in the SCM to the POMs. Precondition is that the previous release descriptor release.properties is still available in the local working copy. You will need to manually remove the tag from your SCM:
git tag -d <tag>
git push --tags -f
  • If the prepare was successful, the release can be performed. First make sure that you have sufficient rights for writing on the target machine by scp.
cd srv/www/staging/projects
chown -R user:group *
where user:group are user and group used for scp. Now enter
mvn release:perform
A new repository is created contained in the release repository on nexus as target for the pom.xml and site information. The name of the new repository contains the tag generated by the maven release prepare build. The system now creates all the needed artifacts (jars, javadoc, etc.) for the release and starts the tests. If these were okay, the artifacts will be transfered to Nexus. For gaining speed direct the output to an text file.


Further information on the release mechanism can be found on the Homepage of the Maven release plugin.

Further steps to do after a release[edit]

  • Copy over the released ear to the qa-machine for testing
  • Clean up the old SNAPSHOT ear on the dev-machine
  • Open a new release in JIRA and close the open release, which you just have released

Publish a release[edit]

  • Copy documentation to pubman.mpdl.mpg.de:/srv/www/projects/pubman/get_pubman
  • Copy also ear and installer to this destination

Hints - Deal with errors[edit]

  • For testing purposes first start the maven build process with -DdryRun = true. For detailed description see http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html
  • If a release fails and the system states that the tag you want to create is already in the svn repository, you have to delete it manually directly in the repository
  • If the maven release:perform fails remove the pom.xml and site information in the corresponding release directory on nexus before calling maven release:perform for a second time. Otherwise the upload on nexus will fail.