Difference between revisions of "AWOB"

From MPDLMediaWiki
Jump to navigation Jump to search
 
(82 intermediate revisions by 5 users not shown)
Line 1: Line 1:
{{Astronomers’ Workbench}}
{{AWOB_NO_TOC_Public}}
<accesscontrol>eSciDoc,, GAVO,,</accesscontrol>
Preparation and planning for shared MPDL project "Scholarly Workbench for Astronomy"


Based on experiences and outcomes of [http://www.g-vo.org/www/ German Virtual Observatory (GAVO)]
[[Category:AWOB]]


Contacts@[http://www.mpe-garching.mpg.de/ MPE]:
== Initiators, Coordinators, the Team, and Partners ==
Jaiwon Kim, Gerard Lemson, Wolfgang Voges


*Project initiators: Dr. Jai Won Kim, Dr. Gerard Lemson, Dr. Wolfgang Voges, Nata&scaron;a Bulatovi&#263;, Ulla Tschida, Malte Dreyer
*Project coordinators (alphabetical order): Dr. Jaiwon Kim, Dr. Wolfgang Voges, Dr. Andreas Vogler
*[[AWOB_Contacts_Public|Team members]]
*Partner
** [http://www.mpe.mpg.de/ Max Planck Institute for Extraterrestrial Physics (MPE)]
** [http://www.mpa-garching.mpg.de/ Max Planck Institute for Astrophysics (MPA)]
** In a later phase all MPIs associated to astrophysics.


== AWOB in Few Words ==


=Scenarios=
AWOB, the Astronomer's Workbench, is a web based collaboration-data evaluation-publication-platform
==Collaborative environment==
which helps scientific working groups of any size to enhance the communication and to share resources,
Enable easy, wiki-like setup of collaborative environment for shared projects. Allow registered users to access the project, the related pages and linked and/or uploaded data. Link the collaborative platform with eSciDoc repository to allow long-term archiving and PIDs for content stored.
data, results, publication texts etc. throughout the whole scientific life cycle.  


Example for shared project workflow (see details in [https://zim01.gwdg.de/repos/smc/trunk/02_Related_Material/02_Projects/GAVO/mpdl20080912.ppt slides (restricted)]:
Resulting e-publications and publications to the Virtual Observatory allow long term archiving of data,
*definition of shared project and objectives
the annotation of metadata as well as easy access of digital outcomes by other users.
*definition of required experiments
*distribution of responsibilities
*tracking of activities and results: set up experiments, run experiments, produce data, postprocess data, analyse data, extract scientific results
*share data, combine results
*produce  publication-ready paper (shared authoring)


The AWOB project has a duration of 3 years and is based on experiences made by the German
Astrophysical Virtual Observatory ([http://www.g-vo.org GAVO]).


===Components of the collaborative work===
== AWOB and the Scientific Life Cycle ==
*Collaborative work is "publication-in-progress" developed in a Wiki environment as output of the research process.
*Collaborative work comprises
**textual components - mostly metadata describing textual part of a collaborative work such as: abstract, title, authors, affiliations, keywords, annotations of sources, references, and its structural information such as:subject headings, body sections etc.
**non-textual components - mostly representative data sets and illustrations to support the scientific results and conclusions which are presented in tables and figures. Example of figures are  images, plots, and diagrams etc.
**integrated external tools - provide links to existing external tools and services for both textual and non-textual components. This facility allows users to work in familiar environments as well as in interactive ways of visualizing and manipulating non-textual components and their underlying data presented in a collaborative work.
====Textual components====
Textual components enable to:
*link to references to preprints, published papers etc.(ADS, arXiv)
*lookup annotated sources i.e. astronomy objects in databases such as Simbad, NED
*describe the collaborative work with metadata which could be used for query such as: authors, title, abstract, keywords
*describe the structure of collaborative work
[Questions(JK) 1. Is a body of each section also
a textual component?
2. Is an equation a textual component?
3. Figure/table caption: could it be a metadata of non-textual component?]


====Non-textual components====
[[File:LRAWOB.jpg|800px|alt=Scientific Life Cycle]] <br><br>
Non-textual components enable to:
Typical scientific lifecycle in astrophysics. The whole lifecycle is supported by AWOB. Copyrigth for the multi wavelength image of the Milky Way image illustrating "correlation with databases": NASA (National Aeronautics and Space Administration),
*visually represent research data and illustrations such as experiment set up. These components could be represented in tables, and figures including images, plots, and diagrams and may have own metadata (e.g. image metadata)
Goddard Space Flight Center. Within this reference you can find the references to the individual observations. Copyright for the M 104 mid infrared image illustrating "data visualization": NASA/JPL-Caltech/R. Kennicutt (University of Arizona), and the SINGS Team
[Question(JK) Could you give some specific examaples of image metadata? Depending on the context types of metadata could be quite different.
For example, is it like the size, and file type of an image or more science related metadata, e.g., observation date, location, etc ?]
:I think in this case we were more thinking on relly image metadata such as size, resolution etc. The observation dates, location etc. would be candidate for the descriptive metadata of the item with which the image is associated. --[[User:Natasab|Natasa]] 08:34, 21 October 2008 (UTC)
*show metadata for an e.g. image
[JK: Please see above]
*invoke external data collection viewer
[Question(JK) Does this mean to invoke a external tool to view the underlying data?]
:Yes, there were some examples provided by you on the last meeting. --[[User:Natasab|Natasa]] 08:34, 21 October 2008 (UTC)
*download data related to the component
*open external tool for visualizing and working with the data
**for tabular data: e.g. TOPCAT
**for image data: e.g. Aladin
**for spectral data: e.g. SpecView, Splat, VOSpec
**[http://www.ivoa.net/Documents/Notes/Plastic/PlasticDesktopInterop-20060601.html PLASTIC enabled]


====Integrated external tools====
== Public Intro to AWOB as a PDF File ==
Integrated external tools enable to link from either textual or non-textual components to existing external astronomical services or tools such as:
*astronomical paper and data archive services - directly linked to widely used publication, and preprint archives such as ADS, and arXiv, and astronomical data archives such as NED, Simbad, VizieR and SkyServer. Also  provide link to registries of astronomical resources which enable users to discover and to get connected to smaller data sets, and services.
*common analysis environments (IDL, ...)
*standardized data retrieval services for various types of  astronomical data  compliant to IVOA standards :
**services for retrieval of image data ( [http://www.ivoa.net/Documents/latest/SIA.html Simple Image Access Protocol SIA] )
**services for retrieval of spectra ([http://www.ivoa.net/Documents/latest/SSA.html Simple Spectral Access Protocol SSA])
**services for retrieval records from source catalogs ([http://www.ivoa.net/Documents/latest/ConeSearch.html Simple Cone Search SCS])
**simulation database SimDB ([http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/IVOATheorySimulationDatamodel Simulation Data model and Simulation data Access Protocol SimDAP])
**invoke queries on external services that support [http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/IvoaVOQL Astronomical Data Query Language(ADQL) ] and Table Access Protocol(TAP). ADQL and TAP enable a general access mechanism for tabular data set not limited astronomical data.


==Sharing of content==
[[{{ns:media}}:2011-06-09.pdf|Download a PDF File]] with a generally understandable public intro to the Astronomer's Workbench. The slides contain supplementary information to this page and some images.
Enable privileged users to upload,  and /or link and describe data with metadata, comments and notes. In astronomy some data are public which are available to everyone, and some are proprietary to a project, or to a group of collaborators.  Usually data from observations become publicly available after a finite time.
It is a common practice for collaborators to share their private data and their analysis in astronomy. In order to share private data, it is typical for a group of collaborators to set up a site with login/password protection, and to exchange data files mostly in FITS(Flexible Image Transport System) format with few description via ftp or xxx. For public data it is possible to access them without special permission(JK: Verify it). Due to the lack of proper metadata capture  it is often difficult to utilize and to query science products and to manage them in the long term unless it is maintained by large data archives. In order to provide consistent and efficient(?) way of sharing data we provide services to upload data with proper metadata which could be used for
querying, and xxxx.  
We support the following data formats:
*Standardised format: FITS, VOTable,
*Custom format: XML(?), Tabular data in comma/tab separated(''more input needed'')


[Question(JK) 1. Could textual and non-textual components of a collaborative work be a sharing content? Somehow I assume that content in this section is limited to astronomical data.
== Situation and Challenge in Today's Astronomy ==
2. Do we need to describe public/private data, as well as raw data/science product somewhere?
3. I put a few points that might be appeared/clarified on this section. Please ignore messiness. ]


Supported astronomical data types are:
Astronomical research is increasingly organised in projects that can include hundreds of collaborators, often spread out over the globe. In collaborations of any size the task of sharing resources such as data products or images and texts for publications can be very cumbersome. The raw data of many observatories have to be evaluated carefully to get reasonable scientific results like calibrated images. Since the programs of such pipelines remain normally unpublished, it is difficult for users not familiar with a certain instrument to mine the data
===Types of Astronomical Data===
*images (from observations, simulations, and etc )
*spectra
*source catalog
*More inputs(Any data which could be in FITS binary table)
*time series
*good time intervals
*light curves
*source extraction(?)


===Auxiliary data types===
== Example of a Typical Astrophysical Quest: A Multi Wavelength Study of NGC 3079 ==
*diagram
*flow chart
*illustration
*publication (textual components)
===Metadata to be supported===
*Bibliographic metadata
**title, author, abstract, subject heading, journal metadata
*Structural metadata/elements
**section/TOC, annotation, footnote, equation, caption, references
*Astronomical metadata
**FITS keywords, values, and comments
**tabular data column name
**xxxx
*Other
**provenance (input files, make files, plotting scripts, analysis code, simulation code, ...)
[(JK) analysis code, and simulation code may be moved to Astronomical metadata]
**log files
**curation (''more input needed'')
**PIDs (ADS, IVOA)
**IVOA standards (VOTable, UCD, UTYPE, Data models, data access protocol, ...)
[Comments(JK) I don't think IVOA standards belong here]


==Shared Authoring==
Multi wavelengths studies of objects allow a holistic view. Often, objects are observed by different groups at different wavelengths. The following images, e.g., combine optical, radio, and X-ray data of the edge-on spiral galaxy NGC 3079. The data reveal hot gas in the halo of the galaxy. The gas is thrown out of the disk into the halo by a super bubble of ionised gas. The images shown are based on the following data: Images with contour plots, image 1 and 2, on optical data (Digital Sky Survey, DSS), X-ray data (Pietsch et al. 1997), and radio & H-&alpha; data (both Veilleux et. al 1994). The false colour image shown in image 3 is based on Hubble Space Telescope and Chandra (X-ray) observations. The copyright belongs to NASA/CXC/STScI/U.North Carolina/G.Cecil.         
Author tools are provided to enable shared and standardised authoring. Authors are supported in developing publication-ready papers.
*Provision of text editor (emacs? TeX IDEs?)
*Import LaTeX article and conversion to html (incl. figures, tables)
*Templates for publication-ready papers (metadata attachments, links, figures, captions)
*allow publication-ready figures from visualisation tools
[Question(JK) 1. How does one distinguish 'publication-ready' from 'publication-in-progress'?
Does publication-ready mean to review the result of 'publication-in-progress' phase?


E-publications, as supported by AWOB, make calibrated images available for all scientists and thereby facilitate multi wavelengths studies.


[[File:NGC3079.jpg|450px|alt=NGC3079]] [[File:NGC3079_b.jpg|165px|alt=NGC3079 HST + X-rays]]


== The Idea of an Astronomer's Workbench ==


The project Astronomer’s Workbench (AWOB) aims to assist astronomers in these tasks by building a web based platform  that will enhance the communication between the scientists and support the centralised organisation and collective usage of resources, ideas, data, results, and documentation.
Use of AWOB should significantly improve coordination of these collaborative projects from their initial formulation to their completion as a publication. An additional focus of AWOB is to facilitate the publication of the project data to the astronomy community, both using Virtual Observatory (VO) standards, and as enhanced e-Publications.


=Related tools/projects=
== Functional Components ==
* [http://eprints.soton.ac.uk/42074/ report] on Virtual Research Communitues
* [http://archer.edu.au ARCHER project]
* [http://test-faces.mpdl.mpg.de/faces/ eSciDoc.Faces] allows publishing of images and respective subsets (albums). Provides technical metadata extraction.
* [http://www.myexperiment.org/ My experiment] Find, use, share objects and workflows, community building. Provides REST interfaces, includes Taverna as Workflow Workbench
* [http://www.uni-lueneburg.de/hyperimage/hyperimage/ HyperImage] for describing, relating details within images
* [http://www.xwiki.org/xwiki/bin/view/Main/WebHome XWiki] Java-based wiki
* [http://argentera.inria.fr/wiki/data/Main/MainHome.jsp SweetWiki] another Java-based wiki
* [http://www.microsoft.com/mscorp/tc/trident.mspx Trident] Microsoft product, a "scientific workflow workbench"
* [http://driver2.dans.knaw.nl/demonstrator/eps/hebrew_q1.xml#aggregation Demonstrator Driver] for enhanced publications (embedded queries within publication, usage of OAI-ORE)
* [http://www.star.bris.ac.uk/~mbt/topcat/sun253/index.html TOPCAT] tool for operations on catalogs and tables
* [http://www.ukoln.ac.uk/projects/ebank-uk/ eBank UK project]
* [http://www.jisc.ac.uk/whatwedo/programmes/digitalrepositories2005/store.aspx Jisc StORE project], seems project wiki is not available


[[Category:Astronomers’ Workbench| ]]
To support the mentioned idea of AWOB, the platform will contain the following main functional components:
 
* Explicit project and user management modules for organizing and coordinating a project and associated collaborators.
* Role based authentication and authorisation modules for restricting access to the project resources to selected users.
* Centralised storage and management of the shared data and other resources for a given project.
* Value-adding tools for resource management tasks such as search, visualisation and analysis.
* A WEB portal providing easy access to external web services, and public archives provided by the astronomy community, where possible using Virtual Observatory standards.
* Meta-data extraction and annotation for publication of finalised data products according to Virtual Observatory standards.
* One-click e-publication of data and the relevant artefacts generated during the project.
 
== AWOB History ==
 
AWOB has grown out of two main developments. On the one hand there is the International Virtual Observatory Alliance. Developments in its German node, GAVO, have lead us to see the importance of centralised organisations of data long before final publication is taking place. Also requests for collaborative environments and self-publication of data were made to us.
On the other hand the MPDL has developed tools dealing with concerning sustainable long term archiving of research data, the integration of external tools and services, and the conception of user centred working environments.
 
Having said this, AWOB is extending the assistance for scientists to the whole scientific life cycle from group-building and data acquisition to e-publications and long term archiving. The implementation of AWOB is thereby supported by the tools and infrastructure already developed by the MPDL.
 
<!--
== Scientific Life Cycle ==
 
AWOB aims at assisting the scientists throughout the scientific life cycle. A life cycle diagram will be available soon.
-->
== Time Schedule ==
 
The project is structured in three phases:
Phase one will focus on building a demonstrator and subsequent first release of the community platform together with two project partners (MPE and MPA). Phase two will focus on active outreach to the astrophysical community within the MPG and other organizations, to gather necessary feedback on extensions, improvements and their priorities. Other potential partner institutes will be invited to join the project and we will interact with the Scholarly Workbench working group for sharing experiences and potential collaborations. In Phase three, resulting in the final AWOB release,  we will focus on the holistic scenario of an enhanced publication, i.e. publishing projects, data, and publications. In addition, we will design and incorporate additional tools and interfaces to external systems and databases and produce the final AWOB release. Some of these extensions will be based on the feedback received during the previous phases. In this last phase particular attention will be given to activities concerning outreach and training to MPG and other external communities. This will enable a larger community to use this collaborative platform or parts of it for their own purposes. Finally a cost model will be developed to ensure long term sustainability of the AWOB community platform.
 
== Demonstrator ==
 
A Liferay based Demonstrator of the AWOB platform was available in Nov 2011. With this Demonstrator, we adressed our associated scientists who performed the first testing of the platform.
 
== First Productive Version AWOB 0.5 ==
 
July 2012: The first productive instance of AWOB (AWOB 0.5) is delivered.
 
== Public Minutes of Meetings ==
 
* [[AWOB_Meeting_2011-05-12|Kick-off meeting (2011-05-12)]]
* [[AWOB_Meeting_2011-06-09|Public Intro (2011-06-09)]]

Latest revision as of 11:36, 19 February 2013


Astronomers’ Workbench

General:
Overview · Usage Scenarios
Contact and support

Internal Pages

Initiators, Coordinators, the Team, and Partners[edit]

AWOB in Few Words[edit]

AWOB, the Astronomer's Workbench, is a web based collaboration-data evaluation-publication-platform which helps scientific working groups of any size to enhance the communication and to share resources, data, results, publication texts etc. throughout the whole scientific life cycle.

Resulting e-publications and publications to the Virtual Observatory allow long term archiving of data, the annotation of metadata as well as easy access of digital outcomes by other users.

The AWOB project has a duration of 3 years and is based on experiences made by the German Astrophysical Virtual Observatory (GAVO).

AWOB and the Scientific Life Cycle[edit]

Scientific Life Cycle

Typical scientific lifecycle in astrophysics. The whole lifecycle is supported by AWOB. Copyrigth for the multi wavelength image of the Milky Way image illustrating "correlation with databases": NASA (National Aeronautics and Space Administration), Goddard Space Flight Center. Within this reference you can find the references to the individual observations. Copyright for the M 104 mid infrared image illustrating "data visualization": NASA/JPL-Caltech/R. Kennicutt (University of Arizona), and the SINGS Team

Public Intro to AWOB as a PDF File[edit]

Download a PDF File with a generally understandable public intro to the Astronomer's Workbench. The slides contain supplementary information to this page and some images.

Situation and Challenge in Today's Astronomy[edit]

Astronomical research is increasingly organised in projects that can include hundreds of collaborators, often spread out over the globe. In collaborations of any size the task of sharing resources such as data products or images and texts for publications can be very cumbersome. The raw data of many observatories have to be evaluated carefully to get reasonable scientific results like calibrated images. Since the programs of such pipelines remain normally unpublished, it is difficult for users not familiar with a certain instrument to mine the data.

Example of a Typical Astrophysical Quest: A Multi Wavelength Study of NGC 3079[edit]

Multi wavelengths studies of objects allow a holistic view. Often, objects are observed by different groups at different wavelengths. The following images, e.g., combine optical, radio, and X-ray data of the edge-on spiral galaxy NGC 3079. The data reveal hot gas in the halo of the galaxy. The gas is thrown out of the disk into the halo by a super bubble of ionised gas. The images shown are based on the following data: Images with contour plots, image 1 and 2, on optical data (Digital Sky Survey, DSS), X-ray data (Pietsch et al. 1997), and radio & H-α data (both Veilleux et. al 1994). The false colour image shown in image 3 is based on Hubble Space Telescope and Chandra (X-ray) observations. The copyright belongs to NASA/CXC/STScI/U.North Carolina/G.Cecil.

E-publications, as supported by AWOB, make calibrated images available for all scientists and thereby facilitate multi wavelengths studies.

NGC3079 NGC3079 HST + X-rays

The Idea of an Astronomer's Workbench[edit]

The project Astronomer’s Workbench (AWOB) aims to assist astronomers in these tasks by building a web based platform that will enhance the communication between the scientists and support the centralised organisation and collective usage of resources, ideas, data, results, and documentation. Use of AWOB should significantly improve coordination of these collaborative projects from their initial formulation to their completion as a publication. An additional focus of AWOB is to facilitate the publication of the project data to the astronomy community, both using Virtual Observatory (VO) standards, and as enhanced e-Publications.

Functional Components[edit]

To support the mentioned idea of AWOB, the platform will contain the following main functional components:

  • Explicit project and user management modules for organizing and coordinating a project and associated collaborators.
  • Role based authentication and authorisation modules for restricting access to the project resources to selected users.
  • Centralised storage and management of the shared data and other resources for a given project.
  • Value-adding tools for resource management tasks such as search, visualisation and analysis.
  • A WEB portal providing easy access to external web services, and public archives provided by the astronomy community, where possible using Virtual Observatory standards.
  • Meta-data extraction and annotation for publication of finalised data products according to Virtual Observatory standards.
  • One-click e-publication of data and the relevant artefacts generated during the project.

AWOB History[edit]

AWOB has grown out of two main developments. On the one hand there is the International Virtual Observatory Alliance. Developments in its German node, GAVO, have lead us to see the importance of centralised organisations of data long before final publication is taking place. Also requests for collaborative environments and self-publication of data were made to us. On the other hand the MPDL has developed tools dealing with concerning sustainable long term archiving of research data, the integration of external tools and services, and the conception of user centred working environments.

Having said this, AWOB is extending the assistance for scientists to the whole scientific life cycle from group-building and data acquisition to e-publications and long term archiving. The implementation of AWOB is thereby supported by the tools and infrastructure already developed by the MPDL.

Time Schedule[edit]

The project is structured in three phases: Phase one will focus on building a demonstrator and subsequent first release of the community platform together with two project partners (MPE and MPA). Phase two will focus on active outreach to the astrophysical community within the MPG and other organizations, to gather necessary feedback on extensions, improvements and their priorities. Other potential partner institutes will be invited to join the project and we will interact with the Scholarly Workbench working group for sharing experiences and potential collaborations. In Phase three, resulting in the final AWOB release, we will focus on the holistic scenario of an enhanced publication, i.e. publishing projects, data, and publications. In addition, we will design and incorporate additional tools and interfaces to external systems and databases and produce the final AWOB release. Some of these extensions will be based on the feedback received during the previous phases. In this last phase particular attention will be given to activities concerning outreach and training to MPG and other external communities. This will enable a larger community to use this collaborative platform or parts of it for their own purposes. Finally a cost model will be developed to ensure long term sustainability of the AWOB community platform.

Demonstrator[edit]

A Liferay based Demonstrator of the AWOB platform was available in Nov 2011. With this Demonstrator, we adressed our associated scientists who performed the first testing of the platform.

First Productive Version AWOB 0.5[edit]

July 2012: The first productive instance of AWOB (AWOB 0.5) is delivered.

Public Minutes of Meetings[edit]