Talk:PubMan Func Spec Easy Submission

UC_PM_EASM_01 upload file in structured format

Status/Schedule

• Status: in design
• Schedule:R3

Motivation

• The user wants to upload a locally created BibTeX file, containing one reference.

Expected outcome

Reference is uploaded to a collection on PubMan.

The item is created on PubMan and can be edited/modified afterwards.

Steps

1. The user chooses a collection where he has depositor privileges
2. The user chooses to upload a file in structured format.
3. The user starts the upload.
4. The system processes the uploaded file, checks for completeness, creates an item and releases it immediately. The use case ends successfully.

Alternatives

4. The user gets an error message, indicating type of error (time out during upload, invalid file, validation rules not met).

4a. User tries the upload again. continue with step 3.

4b. User cancels the upload procedure.

Actors involved

User with depositing rights for at least one collection

Data involved

BibTeX File, structured format. See example file by the AEI.

ToDo's

• provide BibTeX mapping

Constraints

• BibTeX files are idiosyncratically structured; BibTool may help with preprocessing/normalization.
  • e.g. upper and lower case corrections, dissolve macros, make out of something like \"a and ä, etc.
• Basic TeX Parsing is needed to interpret non-ascii characters etc., see for example https://dev.livingreviews.org/projects/epubtk/browser/trunk/ePubTk/lib/bibtexlib.py .
• In BibTeX fields are not repeatable; thus multiple authors need to be parsed from the author field.
• BibTeX allows for different formats of representing an author's name; thus the parser needs to be smart enough to recognize them all. See for example http://search.cpan.org/~gward/Text-BibTeX-0.34/BibTeX/Name.pm

Suggested steps

• Normalize BibTeX with BibTool (resolves macros, may be used to map field names, unifies the syntax).
• Parse the - now normalized - records.
• Allow for/provide a mapping for non-standard fields (and possibly genres).

Please precise better what the mapping should look like - from where to what.--Natasa 10:19, 25 March 2008 (CET)

• Handle substructure of fields
  • Multiple entries in author and keyword fields. (see also http://nwalsh.com/tex/texhelp/bibtx-23.html)
  • (La)TeX encoding for special characters/formulae. (see for example https://dev.livingreviews.org/projects/epubtk/browser/trunk/ePubTk/lib/charmaps/tex2unicode.py)
• Map BibTeX fields/genres (including non-standard ones) to eSciDoc PubItem application profile.

Please provide the mapping (from bibtex standard fields to pubitem application profile). If you have non-standard fields, then please provide examples how they can/should be mapped. For R3 we will have to check if "mapping for non-standard fields via GUI" has also extra other requirements concerning GUI (so far was never stated).--Natasa 10:19, 25 March 2008 (CET)

• Java Tools to check
  • http://jabref.sourceforge.net/
  • http://www-plan.cs.colorado.edu/henkel/stuff/javabib/

Future development

• Upload files in structured format containing more than one reference

see Ingestion

UC_PM_EASM_02 fetch full text by identifier (arXiv)

The Full Text Version can be retrieved with:

• http://arxiv.org/pdf/ +arXiv ID (pdf)
• http://arxiv.org/ps/ +arXiv ID (ps)
• http://arxiv.org/src/ +arXiv ID (tex)
• http://arxiv.org/html/ +arXiv ID (html) Found one example with html (http://arxiv.org/html/cs/9811020)

We don't try to retrieve a fulltext version in html because it is very rare

Status/Schedule

• Status: in design
• Schedule: R3.5

Motivation

• The User wants to submit an Item by using the arXivID

Triggers

• This use case is included by the use case UC_PM_SM_04 fetch metadata from external system

Flow of Events

5a: The system could retrieve a full text version. The file box is filled with the retrieved Info.

As default the system restrieves the pdf version if provided.

• The User can choose to see a preview of the uploaded file
• The User can choose to upload 'all available' files from arXiv
• The User can delete the uploaded file and upload a different File from his own Directory

5b: The system couldn’t retrieve a full text version, no entries in the file box:

• The User gets a message why the upload failed
  • Failure1: There is no fulltext version in arXiv
  • Failure2: Technical Problems when communicating with arXiv (Probably try later or something)
• The User can choose to upload a file from his own Directory

6: Continue with use case UC_PM_SM_02 edit item

The user should not be able to change the name of an uploaded file (uploaded automatically or manually)

Expected outcome

• Easy and fast submission of an Item, the full text version is attached

Actors involved

• Depositor

Data involved

• arXivID

Alternatives

• We don`t upload the file, we only link to the arXiv server (e.g.: http://arxiv.org/pdf/astro-ph/0701495 this url always links to the current version).

This alternativ was discussed and rejected.

General Thoughts

• Should we enhance the (technical) metadata of an item with the information where the item originally was created?

(in this case arXiv)

• We should add something like a progress bar when importing data from another system

• The system must take precautions not to get blocked from arxiv for indiscriminate automated download (see http://arxiv.org/RobotsBeware.html).

Future developments

Default Metadata for an item

Status/Schedule

• Status: in specification
• Schedule:to be defined

• • default content category per genre (specified default MD)
  • default creator roles per genre (specified default MD)
  • default source genre per item genre (specified default MD)
  • default creator role if creator is of type organisation (specified default MD)
  • default affiliation (same as previous)(specified as default on GUI)

Default Metadata for an item means, that in the system a default item template is created, with defaulted metadata. As a start, we should do this as system setting. Future development might include some local definitions of item templates on collection level.

Default Metadata means, that they are pre-populated on the GUI, as a kind of proposal, but can be changed by the user.

Context to collection settings: On collection, the allowed genres are defined. In the default MD setting, the default MD for a certain genre or certain creator role are defined.

TODO:

• define sensible defaults in matrix - where to document the matrix in CoLab
• check dependencies in spec "create item from template", "create new revision"=> we have collection settings (limitation of allowed genres), we have default Metadata. In case an item is used as template, the templated item should "overwrite" the default Metadata, but cannot overwrite the collection setting. (?) --Ulla 13:26, 27 February 2008 (CET)

Genre-specific Metadata

Status/Schedule

• Status: in specification
• Schedule:to be defined

Genre-specific Metadata are bound to a certain application profile and are defined as system setting.

This matrix describes the Metadata elements, which are always OR never OR optionally displayed on the edit mask (in easy submission, in normal submission), dependent on a certain genre type. Optional displayed means, that the user has the option to fill them , if needed, but they are somehow "hidden", as less used. This matrix is needed for GUI design. Genre-specific Metadata are not related to validation rules!

TODO:

• define matrix of genre-specific Metadata (Dimensions: Genre, Metadata or Metadata group. Values: always on Easy Submission, always on Normal Submission, optional on ESM, always on NSM). Documentation in CoLab.
• crosscheck assumptions on genre-specific MD with Early Adopter (using functional prototype)

Functional Prototype

Please check the functional prototype for easy submission

Discussion closed

1) External locator for content: As just learned in Nijmgen, user needs the option to provide an external locator for fulltext. I.e., in addition to upload binary content (= upload file), he needs the option to specify an locator/identifier for the binary content located externally, together with the respective content categorie. This is true for Easy as well as normal submission. This external locator will not be part of Metadata, but modeled in content model.(component?)

External locator now part of prototype --Rupert 11:09, 10 March 2008 (CET)

2) Fetch MD, Step 3: Typo on GUI, short short. In addition, would re-phrase to "...might not cover all fetched Metadata". --Ulla 12:35, 15 February 2008 (CET)

As discussed with Natasa - Step 3 is now view item version, with all metadata visible. Short edit is now only for 'Manual Submission' --Rupert 11:09, 10 March 2008 (CET)

• Abstract prototype

- Step 2 (Select collection): Shouldn't there be a note about having only one collection or more than one?

For Easy Submission there will be only one collection in most cases. If only one collection is available the step is not visible. --Rupert 13:58, 27 February 2008 (CET)

- Typo: "Contiuer and complete"

Done --Rupert 13:58, 27 February 2008 (CET)

- After finishing step 5 there is a decision diamond without a condition. I guess it is the validation, right?

Yes (abstract prototype is done by func team) --Rupert 13:58, 27 February 2008 (CET)

- After this decision one is led to step 1.4? I guess this is a typo, too.

I took this out. --Rupert 13:58, 27 February 2008 (CET)

- Another typo: "sucess message"

Done --Rupert 13:58, 27 February 2008 (CET)

- Step 2.1: I do not understand it: Is the upload and the preview on the same page? I would also appreciate some more information on the preview. Or will this be part of the GUI design? - Yet another typo: "successfull"

The page flow diagram is more detailed here: Editable Preview is after step 4 (manual) or after step 3 (BibTeX/Fetch MD) on a separate page.

• Page flow

- The texts next to "choose collection" are swapped.

This was wrong ... done --Rupert 13:58, 27 February 2008 (CET)

- From "view item version" there is no direct way to submit the item, only to the Edit item mask.

Right! View item version is just a rough preview in this case. Because for the existing "view item version" an item must at lease be in state pending?! Please ask Natasa just to be sure.

Comment Natasa:View item version step according to my understanding was invoked if user decides to preview the item quickly without invoking the Full edit mask. The item is not yet created, but is view-item-version page for VO (value object) of the item only (this means, the Submit action should be available). My comment is also in PageFlow diagram. However, the prototype does not show this, instead on BibTex_Fetch_MD_Step3 it provides two options:a) short short preview and quickly submit (please note that short preview does not show all metadata fetched) b) check or edit all available metadata

• The prototype should not offer Option a) and option b) to be selected by the user, but should automatically invoke "option a)" - which was added with intention to provide "classical view item" no item id, no status information provided (because GUI Team thought it is too much disruption to directly show the full-edit mask as it was agreed originally. Therefore alternative approach was to make the view-item-page composed from the Value object (not retrieved from the FW) and in addition user would be able to "submit" the item (as she is doing it regularly from full edit mask) or go back to "edit" the item - by invoking the full-item edit mask. Therefore, "option a)" is what user automatically gets after Step2. --Natasa 16:05, 3 March 2008 (CET)

Done, Natasa can you please check that? --Rupert 22:12, 3 March 2008 (CET)

• Choose Collection

- Can this be linked to the according colab page?

Done --Rupert 13:58, 27 February 2008 (CET)

Error: it is linked to the "submit item" use case and not to "create item" use case --Natasa 16:05, 3 March 2008 (CET)

• Choose submission method

- Where does "cancel" lead?

Back to the Workspace ... Page Flow is updated. --Rupert 17:09, 27 February 2008 (CET)

Comment Natasa:

• There are misleading labels: In action links on left vertical bar one has "Easy submission". Breadcrumbs say "Short Submission".

Rupert: Breadcrumb is more common now: You are here: Home > Main Function > Sub Function > Action--Rupert 10:19, 4 March 2008 (CET)

• Manual submission step 2

- "content-type" is now "content category"

Must be replaced in every file then. At least for ES and FS it's done now. --Rupert 10:19, 4 March 2008 (CET)

- The design of a file input cannot be influenced by CSS. It only depends on the locale set in the clients browser and on the OS (Windows, Linux, Mac). I will attach some examples. The GUI design has to take this into account. - I guess the red star at "genre" means that this field is mandatory. Why isn't there one at "title"?

Done, I added another asterix to the first line of authors --Rupert 13:58, 27 February 2008 (CET)

Comment Natasa

• please use consistent rule for labeling of fields (e.g. at present one has Upload new File, Content Category, Please Upload a file and define the type of content - here we have a mixture of sometimes camel case sometimes not, also the field label is content category and the message asks for the type of content - misleading)
• would be useful if label "Uploaded" is changed to "File" and if the file-name does not contain the directory name but only the "C:\filename.pdf"

Comment Tobias

• I would prefer to change the buttons for the content category into a radio button group. We agreed that buttons should only be used when actions are triggered. Here the user only makes a choice which content type he wants to use.... So it should better be obvious that he is not triggering an action by simply selecting a content category. --Tschraut 12:11, 4 March 2008 (CET)

OK, as discussed...--Rupert 15:22, 5 March 2008 (CET)

• for uploaded files it would be useful if besides the "trash can" icon one has "editing icon" i.e. to be able to edit the category of the file without having to once again upload the same file for another content category (but that would also require some other extra work probably)

Reorganized now...--Rupert 15:22, 5 March 2008 (CET)

• Back/Next are labels to the arrow or are buttons with the arrow icon? (not clear, preference would be to have it as a button, in a same manner as "cancel")
• Maybe back/next can be right aligned next to each other and cancel button can be left alligned (this way it would not be central button on the form) (valid for all steps)
• missing file visibility for files and information on the file size, mime type after the file has been uploaded

As discussed with Ulla and Nicole file visibility and others are not required. --Rupert 15:22, 5 March 2008 (CET)

in that case will the user know if the file uploaded by the system has size/mime-type recognized as he expect - or that is not needed?--Natasa 16:43, 5 March 2008 (CET)

• Proposal: why not naming "Manual submission" as "Use form for data entry" or smth similar, as manual submission is not clear --Natasa 16:05, 3 March 2008 (CET)

Not sure what a librarian would expect to see here. Perhaps we will know more after the workshop ... There are basically two concerns for the user here: Do I have to fill out something (Manually)? Or can I get data from somewhere else? --Rupert 10:19, 4 March 2008 (CET)

Participants of the workshop were all fine with the term 'Manual Submission' --Rupert 11:26, 10 March 2008 (CET)

To this issue: I assume the idea was to click on a button and upload a file (without need to explicitly specify the content-category) - as it was thought can in 80% cases be defaulted based on the genre-value.

Whatever you decide for GUI at the end. The defaulting is anyway to be resolved for R4.--Natasa 16:43, 5 March 2008 (CET)

If last action is 'upload' I would recommend to label it 'uploaded' (Erwartungskonformität). The handling of directory is browser dependent.

• Manual submission step 3

- Creator names are split up into "Name" and "Family name". I expect this would cause faulty entries, because "Name" often is associated either with the surname or with the full name. IMO "Family name" and "Given name" would be better.

I took 'first name' because during interviews people were not sure about given name (!). --Rupert 13:58, 27 February 2008 (CET)

• consistent labeling needed (currently in the prototype: First Name, Family name, Creator Type - it is "Role" actually)--Natasa 18:07, 3 March 2008 (CET)

- Why should the user enter the number of a author?

If the list contains more authors this can be used to insert the author above. --Rupert 13:58, 27 February 2008 (CET)

• even if this is the case it is not very nice to put numbers in, as the "old" numbers will switch (reorder). Why not simply using arrows up/down for this purpose? --Natasa 18:07, 3 March 2008 (CET)

- Once entered, an author cannot be edited anymore, can he?

OK as discussed with Tobias now. Looks a little bit more complex now. --Rupert 15:22, 5 March 2008 (CET)--Rupert 13:58, 27 February 2008 (CET)

• --Natasa 18:07, 3 March 2008 (CET)it is edit form, right? Why preventing the editing in such a manner? Maybe enabling "edit icon" (same issue as for files in step 1) can explicitly enable the editing of fields for the selected row (and thus making only 1 row editable at a time) of the author. In addition moving up/down of the complete record will not be considered as editing of the row, just re-positioning of the row (in this case no explicit numbering is needed, especially not when creating the author-record).

Comment Tobias: I would like to have the following actions for each author row: edit, remove, up, down. All these actions could be placed in the action column. Even more easier would be to enable all author rows in the table. That means the rows aren't displayed with simple output text but with editable inpunt fields. So you do not need the "edit" link in the action column anymore and users are able to edit autor data faster... --Tschraut 14:37, 4 March 2008 (CET)

As we learned from the interviews, scientists only give the corresponding author and do not deal with lists of authors. --Rupert 15:22, 5 March 2008 (CET)

- If so, there should at least be the possibility to move creators up or down. Otherwise, the following can occur: The user enters 5 authors. Then she recognises that she produced at the first author. Now she has to delete all 5 authors to bring them back into the right order.

(--Natasa 18:07, 3 March 2008 (CET) Agree, see comment above as well)

- Is there a concept for entering authors in a predefined format yet? See http://colab.mpdl.mpg.de/mediawiki/Talk:Providing_Lists_of_Authors#Varieties_of_Lists

This would be wonderful, but lists of Authors are not scheduled for R3 so I took this simple approach. --Rupert 13:58, 27 February 2008 (CET)

• Manual submission step 4

- As it is decided that ONLY the "date published in print" will be asked for, there is no need for a dropdown meny, is there?

Could be a misunderstanding; as far as I know "one" publication date only should be possible which can have several types. The dropdown just contains a dummy entry. --Rupert 17:09, 27 February 2008 (CET)

--Natasa 18:07, 3 March 2008 (CET)According the last functional/GUI meeting, it is only "date published in print" - so no need for a dropdown menu. It would be the case for librarians then to later copy/paste the appropriate date.

OK. --Rupert 22:12, 3 March 2008 (CET)

- Because "Language", "Subject" and "Abstract" follow "Title of source, I as a user would have difficulties to decide if these fields belong to my publication or to its source.

So we put the Title of source below the other fields.--Rupert 17:09, 27 February 2008 (CET)

• --Natasa 18:07, 3 March 2008 (CET)Or one can clearly specify the source as "separate group" visually?

• Bibtex import step 2

- Same for file input as above - If the import was successful, the user is lead to "Bibtex import step 3". What happens, if the import fails?

• --Natasa 18:27, 3 March 2008 (CET)What is the label Metadata source next to the "Provide ID" text?

OK, as discussed now --Rupert 15:24, 5 March 2008 (CET)

• --Natasa 18:27, 3 March 2008 (CET)We have talked that the BibTex file upload should contain the possibility either to upload a file with 1 reference or to directly paste the BibTex reference in a text area field (whereas if a file with 1 reference is uploaded, users see the uploaded reference in a text area field). This is not specified on step 2.

Please see annotation in Axure--Rupert 15:22, 5 March 2008 (CET)

• Breadcrumb "Fetch Metadata" or "Provide Metadata" (consistent labeling needed)
• Back/Next/Cancel button issue (see for Manual submission remarks above)

Moves back to step 2 with a message above (see Page Flow). --Rupert 17:09, 27 February 2008 (CET)

• Bibtex import step 3

- Here and on "Choose submission method" radio buttons are used. The user could save one click if we would use direkt links ?!?

Right, but navigation should be done only with back and next. --Rupert 17:09, 27 February 2008 (CET)

--Natasa 18:27, 3 March 2008 (CET)

• Why it is important to make the navigation only with back/next in "Fetch metadata" and in "choose submission method" pages (imho it can be a valid argument for "manual submission")?
• in fetch metadata step 2 "Back" means "Choose other submission method"
• in manual submission step 2 "Back" means "Choose other submission method" (so either label the button "change submission method" or simply remove the button and allow only for "cancel", as Step 3 from fetch metadata is to be removed anyway)
• In manual submission giving "label" to the step such as "Title/Files", "Creators", "Publication info" and naming the "Back/Next" accordingly may be more "wordy" (of course, this is not generic solution, but it may be worth thinking)
• In general, maybe the selected collection name is worth displaying somewhere (in the breadcrumb or somewhere else on the page) - as users already had to select it in the first place.

Rupert: I would do so only for Full Submission. For ES it would not make sense.--Rupert 22:12, 3 March 2008 (CET)

Please take care of the following: ES does not exclude selection of a collection! In any case there may be several collections to which user can make (ES/FS). Displaying the collection name in that case makes sense probably. --Natasa 16:50, 5 March 2008 (CET)

Use Case

Reference Management Software for Lists of References

Publication Management

Albert Einstein Institute

Curses-Based Editor for BibTeX Bibliography Files

Central European Time

Enhanced Scientific Documentation

Graphical User Interface

Identifier

Portable Document Format

Metadata

Collaboration Laboratory

Reference Management Software for Lists of References

Cascading Style Sheets

Operating System