CoLab Migration 2014

ColabSupport,Admin

This pages is a place of employment for the MediaWiki migration of CoLab, to upgrade MediaWiki as well as to integrate new features, plug-ins and services in CoLab.

=Targets=
 * upgrade MediaWiki software of main wikis to latest version (currently 1.22.4)
 * bring new featues, plug-ins and services into CoLab Grossmann (talk) 12:33, 5 February 2014 (CET)
 * get all things done before summer Grossmann (talk) 12:47, 5 February 2014 (CET)

=Tripping Hazards=
 * the restricted access to (selected) pages must be guarantee; (see "Things we 'must have'") Grossmann (talk) 12:19, 5 February 2014 (CET)
 * all linkings and page adresses must persist (or must have a reliable redirect link) Grossmann (talk) 12:42, 5 February 2014 (CET)

=(New) Features, Plug-Ins and Services= All installed extension are display here. And all available extensions are listed here.

Things we "must have"

 * the restricted access to (selected) pages must be guarantee Grossmann (talk) 12:19, 5 February 2014 (CET)
 * eliminate critical defect by code infiltration and see also Grossmann (talk) 09:52, 6 February 2014 (CET)
 * Change the warning by editing a page. Right now this warning appears: "Please note that all contributions to MPDLMediaWiki are considered to be released under the CreativeCommons Attribution 2.0 MPDLMediaWiki:Copyrights for details)." Is this page MPDLMediaWiki:General_disclaimer meant? Grossmann (talk) 12:19, 11 March 2014 (CET)
 * Delete the old privacy policy from Main Page, see COL-40; it was deleted 2007! Grossmann (talk) 14:24, 13 March 2014 (CET)

Things we "should have"

 * generate User Group overview automatically (and replace the manual administrated overview) Grossmann (talk) 12:06, 5 February 2014 (CET)
 * collect and display (not public) the search words (so that missing categories or else can be recognized) Grossmann (talk) 12:15, 5 February 2014 (CET)
 * set the Central European Time (Jira COL-18); i.e. Recent Changes is CET-1 Grossmann (talk) 12:27, 11 March 2014 (CET)
 * change "Feedback" against FAQs in the left sidebar in "about colab" Grossmann (talk) 11:30, 12 March 2014 (CET) (see MediaWiki:Sidebar)

Things that are "nice to have"

 * get Piwik displayed on Special:Piwik; may be there is a problem with the Piwik extension Grossmann (talk) 12:06, 5 February 2014 (CET)
 * encrypt mail adresse automatically (i.e. CryptX for WordPress), instead of the picture [[Image:At.jpg]] Grossmann (talk) 12:12, 5 February 2014 (CET)
 * is CoLab displayed correctly on other/mobile devices? Grossmann (talk) 12:13, 5 February 2014 (CET) the Main Page is not Grossmann (talk) 12:29, 11 March 2014 (CET)
 * (may be) use the tag cloud (for categories) at the main page Grossmann (talk) 12:16, 5 February 2014 (CET)
 * update the MediaWiki skin like here; a lot of testing would be needed for that Grossmann (talk) 10:07, 6 February 2014 (CET)
 * or change the MediaWiki CSS color, i.e. like the mpg.de Grossmann (talk) 12:34, 11 March 2014 (CET)

Relevant Links

 * Extensions for MediaWiki
 * Extensions Matrix for MediaWiki

Relevant CoLab-Pages

 * Jira CoLab Issues
 * Running CoLab note: due to the new instances structure of CoLab other pages of this wiki describing any system administrative aspects of CoLab like entry points or paths, are possibly out of date and do not hold any more
 * possible CoLab features, especially plug-ins
 * old possible CoLab features
 * old ideas and the discussion about them.
 * very old CoLab News
 * old Plug-In Test

= Migration Status =

Productive Instances
The old productive Colab mediawikis .. .. have been copied and fitted into the new structure layout, as described in Running CoLab successfully (on 2014-02-05 at 18:30).
 * http://colab.mpdl.mpg.de/mediawiki (which is instance mw010, to be migrated to v1.22.4 soon)
 * http://colab.mpdl.mpg.de/twrwiki (which is instance mw022, already migrated to v1.22.4)

This step is the main prerequisite for further migration actions.

Migration Instances
Already migrated instances:
 * http://colab.mpdl.mpg.de/twrwiki-old (which is instance mw020)

Migrated test versions of the productive Colab mediawiki have been made accessible under
 * http://colab.mpdl.mpg.de/mediawiki-new (which is instance mw012)

They hold the following status:
 * fresh install of the latest Mediawiki software, i.e. version 1.22.4
 * slightly customized sample configuration file (no longer based on old one):
 * site name, database access, logo location
 * mandatory database update (to fit new software version)
 * activated standard extensions that came with the new mediawiki version:
 * Cite, ImageMap, SyntaxHighlight, WikiCategoryTagCloud
 * added extra extensions that had been been used already:
 * CategoryTree, ConfirmAccount, Contributors, Lingo, NewestPages, UserMerge
 * Piwik, GoogleSiteSearch (mediawiki* only), ContactPage (twrwiki* only)
 * GroupBasedAccessControl (with its original extension code modified slightly)

Unless stated otherwise, all extension codes have not been been modified. Extension configuration has been limited to variable settings in LocalSettings.php only to enable future easy extension upgrades.

Note: Page changes and their expected results made to the productive instances resp. their databases will show up in the migration instances after the next database clone and reload. It is not helpful make mass edit and change actions to the migration instance(s).

Known Issues with GroupBasedAccessControl Extension
The following issues have been found to exist with the GroupBasedAccessControl extension:

Access control lists ending in an empty group ,, have been found to cause problems: Such pages will not get displayed.

Pages will not get displayed - and cannot be edited or submitted - either, if the groups in question to not exist or will be removed later.

Existing occurrences of this error must be corrected. The correction can be done globally for all occurrences in the affected wiki by doing a global search-and-replace action on the database dump of the productive instance prior to being loaded into the migration instance.

Status: This issue has been resolved in the current migration test instances and will be considered for any migration coming.

Note: This issue potentially still exists in the production instances.

See for this the Jira ticket: https://jira.mpdl.mpg.de/browse/COL-44

Next Steps
Make the migration test instances behave exactly like their productive counterparts:
 * install and activate (and configure if needed) the latest versions of necessary extensions
 * adjust more special configuration settings (e.g. access policy)

Once the migration instances are ready to take over the following steps have to be performed for the migration instances to become the productive instances: disable colab's apache configuration colab.conf and reload apache re-enable colab's modified apache configuration and reload apache
 * stop Colab:
 * to reflect the swap edit:
 * colab's apache configuration colab.conf
 * perform the migration swap itself
 * other files referencing these instances in some way:
 * like vm53:/srv/web/colab/.info
 * start colab:

At this point the migration itself is done.

Migration Swap
The swap itself will be a merge of the content data - i.e. database and images directory of instance 'mw0x0' - of the productive wiki with the software data - i.e. configuration file and mediawiki software of instance mw0x1 - of the migration test wiki into a third mediawiki instance 'mw0x2', which will serve as the new, migrated productive wiki.

For the Colab mediawiki the placeholder "x" is 1, for the Colab twrwiki "x" is 2.

So for the Colab mediawiki e.g. the complete migration consists of the following steps:
 * edit the wikis' configuration files:
 * disable further write access to the productive mediawiki 'mw010' by setting the wiki ReadOnly:

 $wgReadOnly = 'This wiki is currently being upgraded to a newer software version.'; 
 * in the configuration of 'mw010' rename '/mediawiki' to '/mediawiki-old'
 * in the configuration of 'mw012' rename '/mediawiki-new' to '/mediawiki'

go to /mw012/maintenance and call  
 * copy the productive wiki's data content:
 * copy the images directory 'mw010/images' to 'mw012/'
 * dump the database of 'mw010', possibly edit the dump (see below) and import it to become the database of 'mw012'
 * update the imported database of 'mw012':
 * 1) php update.php
 * 2) php rebuildLocalisationCache.php --force


 * update apache
 * rename '/mediawiki' to '/mediawiki-old' and '/mediawiki-new' to '/mediawiki' in the apache vhost configuration
 * restart apache

Stream editing the database dump using sed</tt> could consider:
 * renaming '/srv/web/colab/inst/mw0x0' to '/srv/web/colab/inst/mw0x2'
 * perform global rename actions, where and if necessary

For the colab mediawikis: Note, that it is neither necessary nor useful to rename URLs (as it might be done for creating mw0x0).
 * sed -e 's/\,\, *\(<\/accesscontrol>\)/\1/g' to accomplish the GroupBasedAccessControl issue
 * sed -e 's?\(table\|tables\|TABLE\|TABLES\|EXISTS\|INTO\) `twr_?\1 `?g' to get rid of non-empty wgDBprefix settings (here twr_)

Once all migrations have been done the following situation holds: (from internal networks only)
 * the migrated wikis are accessible via 'mediawiki == mw012' resp. 'twrwiki == mw022'
 * the former productive wikis are accessible via 'mediawiki-old == mw010' resp. 'twrwiki-old == mw020'
 * the former migration test wikis 'mediawiki1 == mw011' resp. 'twrwiki1 == mw021' became obsolete

After some time all references to the old wiki migration test and productive instances may be deleted. If a new migration has to be performed in the future, the same cycle may start over again.

Note: there is no need to swap 'mediawiki' resp. 'twrwiki' at the same time if you really want to do so, see the database part of the chapter describing how to clone an instance
 * each wiki may be treated separately
 * the instance names and locations 'mw*' do not have be changed
 * in and after the final phase of migration, the remaining mw0x1 migration test instance may be removed and 'mw0x2' can be used directly to become a swappable clone of instance 'mw0x0'

=Keep in Mind=
 * generating RSS-feeds of a page is implemented in MediaWiki (at the page history appears in the toolbox on the left the point "atom") Grossmann (talk) 13:23, 5 February 2014 (CET)
 * this is the list of active user (inform them before working at the server) Grossmann (talk) 13:42, 5 February 2014 (CET)