Difference between revisions of "PubMan Indexing Problems"

From MPDLMediaWiki
Jump to navigation Jump to search
 
(16 intermediate revisions by 2 users not shown)
Line 4: Line 4:
=Fehlerhistorie=
=Fehlerhistorie=
*16.01.13
*16.01.13
**14:59
**14:59 Uhr
**15:57
**15:57 Uhr
*17.01.13
*17.01.13
**11:11
**11:11 Uhr
*18.01.13
*18.01.13
**12:23
**12:23 Uhr
*22.01.13
*22.01.13
**15:18
**15:18 Uhr
**16:14
**16:14 Uhr
*23.01.13
*23.01.13
**13:19
**13:19 Uhr
**17:06 Uhr
*24.01.13
**17:06 Uhr
*28.01.13
**15:46 Uhr
*29.02.13
**11:25 Uhr
*30.01.13
**11:35 Uhr
*31.01.13
**13:24 Uhr
**17:33 Uhr
*01.02.13
**08:45 Uhr
**09:39 Uhr
*04.02.2013
**11:10 Uhr
*06.02.2013
**09:33 Uhr
**10:56 Uhr
**11:48 Uhr
**13:48 Uhr
**14:25 Uhr
**22:58 Uhr


=Infos=
=Infos=
Line 63: Line 87:
*Fehler von LIVE (wurde so auch auf QA reproduziert bei zwei gleichzeitigen Importen - auf LIVE war kein Import aktiv wenn der Fehler aufgetreten ist)
*Fehler von LIVE (wurde so auch auf QA reproduziert bei zwei gleichzeitigen Importen - auf LIVE war kein Import aktiv wenn der Fehler aufgetreten ist)
:<code>ERROR 2013-01-17 11:11:37,305 (RESTImpl) dk.defxws.fedoragsearch.server.errors.GenericSearchException: IndexReader close error indexName=item_container_admin :; nested exception is: org.apache.lucene.index.CorruptIndexException: failed to locate current segments_N file</code>
:<code>ERROR 2013-01-17 11:11:37,305 (RESTImpl) dk.defxws.fedoragsearch.server.errors.GenericSearchException: IndexReader close error indexName=item_container_admin :; nested exception is: org.apache.lucene.index.CorruptIndexException: failed to locate current segments_N file</code>
*Reproduziert durch batch-delete auf QA und batch-imoport auf DEV
*Reproduziert durch batch-delete auf QA und batch-import auf DEV
:<code>2013-01-17 13:29:32,371 (RESTImpl) dk.defxws.fedoragsearch.server.errors.GenericSearchException: updateIndex deletePid error indexName=item_container_admin pid=escidoc:1594805:LV; nested exception is: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/coreservice/server/default/data/index/lucene/item_container_admin/write.lock</code>
:<code>2013-01-17 13:29:32,371 (RESTImpl) dk.defxws.fedoragsearch.server.errors.GenericSearchException: updateIndex deletePid error indexName=item_container_admin pid=escidoc:1594805:LV; nested exception is: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/coreservice/server/default/data/index/lucene/item_container_admin/write.lock</code>


Line 404: Line 428:
   at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:222)</syntaxhighlight>
   at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:222)</syntaxhighlight>


== Experimente ==
*fgsindex.defaultWriteLockTimeout = 3000 (msec anstelle von 100.000.000) wait time for an IndexWriter for the same directory in case of a write.lock before throwing a LockObtainFailedException
<syntaxhighlight lang="java">
de.mpg.escidoc.services.common.exceptions.TechnicalException: <exception>
  <title><h1>500 Internal eSciDoc System Error</h1></title>
  <message><p>&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;
&lt;resultPage&gt;
&lt;error&gt;
&lt;message&gt;Update error pidOrFilename=escidoc:1602273; nested exception is:
        dk.defxws.fedoragsearch.server.errors.GenericSearchException: IndexWriter new error, creating index indexName=item_container_admin :
; nested exception is:
        org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/coreservice/server/default/data/index/lucene/item_container_admin/write.lock&lt;/message&gt;
&lt;/error&gt;
&lt;/resultPage&gt;
</p></message>
  <class><p>de.escidoc.core.common.exceptions.system.SystemException</p></class>
  <stack-trace><p><![CDATA[
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody8(IndexingHandler.java:502)
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody9$advice(IndexingHandler.java:57)
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing(IndexingHandler.java:1)
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody6(IndexingHandler.java:338)
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody7$advice(IndexingHandler.java:57)
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing(IndexingHandler.java:1)
    de.escidoc.core.common.business.indexing.IndexingHandler.indexResource(IndexingHandler.java:285)
    de.escidoc.core.common.business.indexing.IndexingHandler.addResource(IndexingHandler.java:228)
    de.escidoc.core.common.business.indexing.IndexingHandler.resourceCreated_aroundBody0(IndexingHandler.java:153)
    de.escidoc.core.common.business.indexing.IndexingHandler.resourceCreated_aroundBody1$advice(IndexingHandler.java:57)
    de.escidoc.core.common.business.indexing.IndexingHandler.resourceCreated(IndexingHandler.java:1)
    de.escidoc.core.om.business.fedora.item.ItemResourceListener.fireItemCreated(ItemResourceListener.java:98)
    de.escidoc.core.om.business.fedora.item.FedoraItemHandler.create_aroundBody6(FedoraItemHandler.java:434)
    de.escidoc.core.om.business.fedora.item.FedoraItemHandler$AjcClosure7.run(FedoraItemHandler.java:1)
    org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
    de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
    de.escidoc.core.om.business.fedora.item.FedoraItemHandler.create(FedoraItemHandler.java:409)
    de.escidoc.core.om.business.fedora.item.FedoraItemHandler$$FastClassByCGLIB$$575498c2.invoke(<generated>)
    net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191)
    org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
    org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    de.escidoc.core.om.performance.PerformanceStatisticsInterceptor.invoke_aroundBody2(PerformanceStatisticsInterceptor.java:89)
    de.escidoc.core.om.performance.PerformanceStatisticsInterceptor$AjcClosure3.run(PerformanceStatisticsInterceptor.java:1)
    org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
    de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
    de.escidoc.core.om.performance.PerformanceStatisticsInterceptor.invoke(PerformanceStatisticsInterceptor.java:80)
</syntaxhighlight>
*project FgsLucene OperationsImpl.java ohne ''closeIndexReader''
<syntaxhighlight lang="java">
ERROR 2013-02-12 08:35:14,415 [http-0.0.0.0-8080-8] (RESTImpl) RESTImpl error:
dk.defxws.fedoragsearch.server.errors.GenericSearchException: IndexReader open error indexName=item_container_admin :
; nested exception is:
        java.io.FileNotFoundException: /data/coreservice/server/default/data/index/lucene/item_container_admin/_7690.prx (Too many open files)
        at dk.defxws.fgslucene.OperationsImpl.getIndexReader(OperationsImpl.java:679)
        at dk.defxws.fgslucene.OperationsImpl.updateIndex(OperationsImpl.java:265)
        at dk.defxws.fedoragsearch.server.GenericOperationsImpl.updateIndex(GenericOperationsImpl.java:356)
        at dk.defxws.fedoragsearch.server.RESTImpl.updateIndex(RESTImpl.java:280)
        at dk.defxws.fedoragsearch.server.RESTImpl.doGet(RESTImpl.java:117)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
        at java.lang.Thread.run(Thread.java:662)
</syntaxhighlight>
 


[[Category:PubMan_Development]]
[[Category:PubMan_Development]]

Latest revision as of 07:47, 12 February 2013

Beschreibung[edit]

item_container_admin wird mit einer write.lock-Datei permanent geblockt

Fehlerhistorie[edit]

  • 16.01.13
    • 14:59 Uhr
    • 15:57 Uhr
  • 17.01.13
    • 11:11 Uhr
  • 18.01.13
    • 12:23 Uhr
  • 22.01.13
    • 15:18 Uhr
    • 16:14 Uhr
  • 23.01.13
    • 13:19 Uhr
    • 17:06 Uhr
  • 24.01.13
    • 17:06 Uhr
  • 28.01.13
    • 15:46 Uhr
  • 29.02.13
    • 11:25 Uhr
  • 30.01.13
    • 11:35 Uhr
  • 31.01.13
    • 13:24 Uhr
    • 17:33 Uhr
  • 01.02.13
    • 08:45 Uhr
    • 09:39 Uhr
  • 04.02.2013
    • 11:10 Uhr
  • 06.02.2013
    • 09:33 Uhr
    • 10:56 Uhr
    • 11:48 Uhr
    • 13:48 Uhr
    • 14:25 Uhr
    • 22:58 Uhr

Infos[edit]

  • Scheint eher reproduzierbar mit sehr einfachen Items
  • Sicherheits-Alarm kommt mit 5 Min Verzögerung
  • Timestamp wird für jedes File geändert(evtl. Alarm auf Timestampänderungen eingrenzen?)
  • Framework 1.3.8 verwendet Lucene 3.4.0

Mögliche Fehlergründe[edit]

laut:

http://www.gossamer-threads.com/lists/lucene/java-dev/98176

könnte ein NFS (Network File System) stale

A filehandle becomes stale whenever the file or directory referenced by the handle is removed by another host, while your client still holds an active reference to the object. A typical example occurs when the current directory of a process, running on your client, is removed on the server (either by a process running on the server or on another client).

das Problem sein. Mehr zu NFS stales unter:

http://www.cyberciti.biz/tips/nfs-stale-file-handle-error-and-solution.html
https://issues.apache.org/jira/browse/LUCENE-1011

Aus org.apache.lucene.index.IndexFileDeleter : 

if (currentCommitPoint == null) {
    // We did not in fact see the segments_N file
    // corresponding to the segmentInfos that was passed
    // in.  Yet, it must exist, because our caller holds
    // the write lock.  This can happen when the directory
    // listing was stale (eg when index accessed via NFS
    // client with stale directory listing cache).  So we
    // try now to explicitly open this commit point:
    SegmentInfos sis = new SegmentInfos();
    try {
        sis.read(directory, segmentInfos.getCurrentSegmentFileName(), codecs);
    } catch (IOException e) {
        throw new CorruptIndexException("failed to locate current segments_N file");
    }
    if (infoStream != null)
    message("forced open of current segments file " + segmentInfos.getCurrentSegmentFileName());
    currentCommitPoint = new CommitPoint(commitsToDelete, directory, sis);
    commits.add(currentCommitPoint);
    incRef(sis, true);
}

Problem: wir verwenden kein NFS. Da dies aber die einzige Stelle in Lucene zu sein scheint, die diese Exception wirft, muss der Fehler auf das System wie ein NFS stale wirkt.

Logfiles[edit]

fedoragsearch.log[edit]

  • Fehler von LIVE (wurde so auch auf QA reproduziert bei zwei gleichzeitigen Importen - auf LIVE war kein Import aktiv wenn der Fehler aufgetreten ist)
ERROR 2013-01-17 11:11:37,305 (RESTImpl) dk.defxws.fedoragsearch.server.errors.GenericSearchException: IndexReader close error indexName=item_container_admin :; nested exception is: org.apache.lucene.index.CorruptIndexException: failed to locate current segments_N file
  • Reproduziert durch batch-delete auf QA und batch-import auf DEV
2013-01-17 13:29:32,371 (RESTImpl) dk.defxws.fedoragsearch.server.errors.GenericSearchException: updateIndex deletePid error indexName=item_container_admin pid=escidoc:1594805:LV; nested exception is: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/coreservice/server/default/data/index/lucene/item_container_admin/write.lock

indexer_error.log[edit]

  • Fehler von LIVE
ERROR 2013-01-14 17:33:14,334 Error on indexing resource escidoc:1481521, index null, all indexes true, action update:
<resultPage>
    <error>
        <message>
        IndexReader close error indexName=item_container_admin :; nested exception is:org.apache.lucene.index.CorruptIndexException: failed to locate current segments_N file
        </message>
    </error>
</resultPage>

server.log[edit]

  • keine passende Meldung, erst die Folgeerscheinungen sind sichtbar

import-log[edit]

  • Warning im Import-Bereich:
TechnicalException: <exception>
<title><h1>500 Internal Application Server Error</h1></title>
<message><p>HTTP connection to &quot;http://localhost:8080/fedoragsearch/rest?operation=updateIndex&amp;action=deletePid&amp;repositoryName=escidocrepository&amp;indexName=content_model_admin&amp;value=escidoc:1594776:LV&quot; failed: &lt;html&gt;&lt;head&gt;&lt;title&gt;JBossWeb/2.0.1.GA - Error report&lt;/title&gt;&lt;style&gt;&lt;!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--&gt;&lt;/style&gt; &lt;/head&gt;&lt;body&gt;&lt;h1&gt;HTTP Status 500 - &lt;/h1&gt;&lt;HR size=&quot;1&quot; noshade=&quot;noshade&quot;&gt;&lt;p&gt;&lt;b&gt;type&lt;/b&gt; Exception report&lt;/p&gt;&lt;p&gt;&lt;b&gt;message&lt;/b&gt; &lt;u&gt;&lt;/u&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;description&lt;/b&gt; &lt;u&gt;The server encountered an internal error () that prevented it from fulfilling this request.&lt;/u&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;exception&lt;/b&gt; &lt;pre&gt;dk.defxws.fedoragsearch.server.errors.ConfigException: get stylesheet from url search/config/rest/:
; nested exception is: 
  dk.defxws.fedoragsearch.server.errors.ConfigException: /search/config/rest/.xslt not found
  dk.defxws.fedoragsearch.server.GTransformer.getTransformer(GTransformer.java:97)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:126)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:119)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:189)
  dk.defxws.fedoragsearch.server.RESTImpl.doGet(RESTImpl.java:154)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
  org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
&lt;/pre&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;root cause&lt;/b&gt; &lt;pre&gt;dk.defxws.fedoragsearch.server.errors.ConfigException: /search/config/rest/.xslt not found
  dk.defxws.fedoragsearch.server.GTransformer.getTransformer(GTransformer.java:69)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:126)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:119)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:189)
  dk.defxws.fedoragsearch.server.RESTImpl.doGet(RESTImpl.java:154)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
  org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
&lt;/pre&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;note&lt;/b&gt; &lt;u&gt;The full stack trace of the root cause is available in the JBossWeb/2.0.1.GA logs.&lt;/u&gt;&lt;/p&gt;&lt;HR size=&quot;1&quot; noshade=&quot;noshade&quot;&gt;&lt;h3&gt;JBossWeb/2.0.1.GA&lt;/h3&gt;&lt;/body&gt;&lt;/html&gt;
</p></message>
<class><p>de.escidoc.core.common.exceptions.system.ApplicationServerSystemException</p></class>
<stack-trace><p><![CDATA[
de.escidoc.core.common.business.indexing.GsearchHandler.requestDeletion_aroundBody2(GsearchHandler.java:205)
de.escidoc.core.common.business.indexing.GsearchHandler.requestDeletion_aroundBody3$advice(GsearchHandler.java:57)
de.escidoc.core.common.business.indexing.GsearchHandler.requestDeletion(GsearchHandler.java:1)
de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody6(IndexingHandler.java:344)
de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody7$advice(IndexingHandler.java:57)
de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing(IndexingHandler.java:1)
de.escidoc.core.common.business.indexing.IndexingHandler.deleteResource(IndexingHandler.java:243)
de.escidoc.core.common.business.indexing.IndexingHandler.resourceDeleted_aroundBody2(IndexingHandler.java:174)
de.escidoc.core.common.business.indexing.IndexingHandler.resourceDeleted_aroundBody3$advice(IndexingHandler.java:57)
de.escidoc.core.common.business.indexing.IndexingHandler.resourceDeleted(IndexingHandler.java:1)
de.escidoc.core.om.business.fedora.item.ItemResourceListener.fireItemDeleted(ItemResourceListener.java:171)
de.escidoc.core.om.business.fedora.item.FedoraItemHandler.delete_aroundBody10(FedoraItemHandler.java:479)
de.escidoc.core.om.business.fedora.item.FedoraItemHandler$AjcClosure11.run(FedoraItemHandler.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
de.escidoc.core.om.business.fedora.item.FedoraItemHandler.delete(FedoraItemHandler.java:474)
de.escidoc.core.om.business.fedora.item.FedoraItemHandler$$FastClassByCGLIB$$575498c2.invoke(<generated>)
net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191)
org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
de.escidoc.core.om.performance.PerformanceStatisticsInterceptor.invoke_aroundBody2(PerformanceStatisticsInterceptor.java:89)
de.escidoc.core.om.performance.PerformanceStatisticsInterceptor$AjcClosure3.run(PerformanceStatisticsInterceptor.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
de.escidoc.core.om.performance.PerformanceStatisticsInterceptor.invoke(PerformanceStatisticsInterceptor.java:80)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
de.escidoc.core.om.business.fedora.item.FedoraItemHandler$$EnhancerByCGLIB$$f3ee31b5.delete(<generated>)
de.escidoc.core.om.service.ItemHandler.delete_aroundBody4(ItemHandler.java:123)
de.escidoc.core.om.service.ItemHandler$AjcClosure5.run(ItemHandler.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
de.escidoc.core.om.service.ItemHandler.delete(ItemHandler.java:119)
de.escidoc.core.om.ejb.ItemHandlerBean.delete_aroundBody32(ItemHandlerBean.java:154)
de.escidoc.core.om.ejb.ItemHandlerBean$AjcClosure33.run(ItemHandlerBean.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.aa.security.aop.SecurityInterceptor.proceed(SecurityInterceptor.java:237)
de.escidoc.core.aa.security.aop.SecurityInterceptor.ajc$inlineAccessMethod$de_escidoc_core_aa_security_aop_SecurityInterceptor$de_escidoc_core_aa_security_aop_SecurityInterceptor$proceed(SecurityInterceptor.java:1)
de.escidoc.core.aa.security.aop.SecurityInterceptor.authorize(SecurityInterceptor.java:206)
de.escidoc.core.om.ejb.ItemHandlerBean.delete_aroundBody34(ItemHandlerBean.java:1)
de.escidoc.core.om.ejb.ItemHandlerBean$AjcClosure35.run(ItemHandlerBean.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.common.util.aop.StatisticInterceptor.proceed(StatisticInterceptor.java:205)
de.escidoc.core.common.util.aop.StatisticInterceptor.ajc$inlineAccessMethod$de_escidoc_core_common_util_aop_StatisticInterceptor$de_escidoc_core_common_util_aop_StatisticInterceptor$proceed(StatisticInterceptor.java:1)
de.escidoc.core.common.util.aop.StatisticInterceptor.createStatisticRecord(StatisticInterceptor.java:145)
de.escidoc.core.om.ejb.ItemHandlerBean.delete_aroundBody36(ItemHandlerBean.java:154)
de.escidoc.core.om.ejb.ItemHandlerBean$AjcClosure37.run(ItemHandlerBean.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
de.escidoc.core.om.ejb.ItemHandlerBean.delete(ItemHandlerBean.java:144)
sun.reflect.GeneratedMethodAccessor614.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.jboss.invocation.Invocation.performCall(Invocation.java:359)
org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:237)
org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:158)
org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:169)
org.jboss.ejb.plugins.CallValidationInterceptor.invoke(CallValidationInterceptor.java:63)
org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:121)
org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:350)
org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:181)
org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:168)
org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:205)
org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:138)
org.jboss.ejb.SessionContainer.internalInvoke(SessionContainer.java:648)
org.jboss.ejb.Container.invoke(Container.java:960)
sun.reflect.GeneratedMethodAccessor404.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
org.jboss.invocation.local.LocalInvoker$MBeanServerAction.invoke(LocalInvoker.java:169)
org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:118)
org.jboss.invocation.InvokerInterceptor.invokeLocal(InvokerInterceptor.java:209)
org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:195)
org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:61)
org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:70)
org.jboss.proxy.ejb.StatelessSessionInterceptor.invoke(StatelessSessionInterceptor.java:112)
org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:100)
$Proxy160.delete(Unknown Source)
sun.reflect.GeneratedMethodAccessor613.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.springframework.remoting.rmi.RmiClientInterceptorUtils.invokeRemoteMethod(RmiClientInterceptorUtils.java:111)
org.springframework.ejb.access.SimpleRemoteSlsbInvokerInterceptor.doInvoke(SimpleRemoteSlsbInvokerInterceptor.java:99)
org.springframework.ejb.access.AbstractRemoteSlsbInvokerInterceptor.invokeInContext(AbstractRemoteSlsbInvokerInterceptor.java:141)
org.springframework.ejb.access.AbstractSlsbInvokerInterceptor.invoke(AbstractSlsbInvokerInterceptor.java:189)
de.escidoc.core.common.util.service.RemoteStatelessEjbProxyFactoryBean.invoke_aroundBody2(RemoteStatelessEjbProxyFactoryBean.java:171)
de.escidoc.core.common.util.service.RemoteStatelessEjbProxyFactoryBean.invoke_aroundBody3$advice(RemoteStatelessEjbProxyFactoryBean.java:57)
de.escidoc.core.common.util.service.RemoteStatelessEjbProxyFactoryBean.invoke(RemoteStatelessEjbProxyFactoryBean.java:1)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
$Proxy253.delete(Unknown Source)
sun.reflect.GeneratedMethodAccessor612.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
java.lang.Thread.run(Thread.java:662)
]]></p></stack-trace>
</exception>
  at de.mpg.escidoc.services.pubman.exceptions.ExceptionHandler.handleException(ExceptionHandler.java:93)
AxisFault: <exception>
<title><h1>500 Internal Application Server Error</h1></title>
<message><p>HTTP connection to &quot;http://localhost:8080/fedoragsearch/rest?operation=updateIndex&amp;action=deletePid&amp;repositoryName=escidocrepository&amp;indexName=content_model_admin&amp;value=escidoc:1594776:LV&quot; failed: &lt;html&gt;&lt;head&gt;&lt;title&gt;JBossWeb/2.0.1.GA - Error report&lt;/title&gt;&lt;style&gt;&lt;!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--&gt;&lt;/style&gt; &lt;/head&gt;&lt;body&gt;&lt;h1&gt;HTTP Status 500 - &lt;/h1&gt;&lt;HR size=&quot;1&quot; noshade=&quot;noshade&quot;&gt;&lt;p&gt;&lt;b&gt;type&lt;/b&gt; Exception report&lt;/p&gt;&lt;p&gt;&lt;b&gt;message&lt;/b&gt; &lt;u&gt;&lt;/u&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;description&lt;/b&gt; &lt;u&gt;The server encountered an internal error () that prevented it from fulfilling this request.&lt;/u&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;exception&lt;/b&gt; &lt;pre&gt;dk.defxws.fedoragsearch.server.errors.ConfigException: get stylesheet from url search/config/rest/:
; nested exception is: 
  dk.defxws.fedoragsearch.server.errors.ConfigException: /search/config/rest/.xslt not found
  dk.defxws.fedoragsearch.server.GTransformer.getTransformer(GTransformer.java:97)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:126)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:119)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:189)
  dk.defxws.fedoragsearch.server.RESTImpl.doGet(RESTImpl.java:154)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
  org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
&lt;/pre&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;root cause&lt;/b&gt; &lt;pre&gt;dk.defxws.fedoragsearch.server.errors.ConfigException: /search/config/rest/.xslt not found
  dk.defxws.fedoragsearch.server.GTransformer.getTransformer(GTransformer.java:69)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:126)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:119)
  dk.defxws.fedoragsearch.server.GTransformer.transform(GTransformer.java:189)
  dk.defxws.fedoragsearch.server.RESTImpl.doGet(RESTImpl.java:154)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
  javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
  org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
&lt;/pre&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;note&lt;/b&gt; &lt;u&gt;The full stack trace of the root cause is available in the JBossWeb/2.0.1.GA logs.&lt;/u&gt;&lt;/p&gt;&lt;HR size=&quot;1&quot; noshade=&quot;noshade&quot;&gt;&lt;h3&gt;JBossWeb/2.0.1.GA&lt;/h3&gt;&lt;/body&gt;&lt;/html&gt;
</p></message>
<class><p>de.escidoc.core.common.exceptions.system.ApplicationServerSystemException</p></class>
<stack-trace><p><![CDATA[
de.escidoc.core.common.business.indexing.GsearchHandler.requestDeletion_aroundBody2(GsearchHandler.java:205)
de.escidoc.core.common.business.indexing.GsearchHandler.requestDeletion_aroundBody3$advice(GsearchHandler.java:57)
de.escidoc.core.common.business.indexing.GsearchHandler.requestDeletion(GsearchHandler.java:1)
de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody6(IndexingHandler.java:344)
de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody7$advice(IndexingHandler.java:57)
de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing(IndexingHandler.java:1)
de.escidoc.core.common.business.indexing.IndexingHandler.deleteResource(IndexingHandler.java:243)
de.escidoc.core.common.business.indexing.IndexingHandler.resourceDeleted_aroundBody2(IndexingHandler.java:174)
de.escidoc.core.common.business.indexing.IndexingHandler.resourceDeleted_aroundBody3$advice(IndexingHandler.java:57)
de.escidoc.core.common.business.indexing.IndexingHandler.resourceDeleted(IndexingHandler.java:1)
de.escidoc.core.om.business.fedora.item.ItemResourceListener.fireItemDeleted(ItemResourceListener.java:171)
de.escidoc.core.om.business.fedora.item.FedoraItemHandler.delete_aroundBody10(FedoraItemHandler.java:479)
de.escidoc.core.om.business.fedora.item.FedoraItemHandler$AjcClosure11.run(FedoraItemHandler.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
de.escidoc.core.om.business.fedora.item.FedoraItemHandler.delete(FedoraItemHandler.java:474)
de.escidoc.core.om.business.fedora.item.FedoraItemHandler$$FastClassByCGLIB$$575498c2.invoke(<generated>)
net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191)
org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
de.escidoc.core.om.performance.PerformanceStatisticsInterceptor.invoke_aroundBody2(PerformanceStatisticsInterceptor.java:89)
de.escidoc.core.om.performance.PerformanceStatisticsInterceptor$AjcClosure3.run(PerformanceStatisticsInterceptor.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
de.escidoc.core.om.performance.PerformanceStatisticsInterceptor.invoke(PerformanceStatisticsInterceptor.java:80)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
de.escidoc.core.om.business.fedora.item.FedoraItemHandler$$EnhancerByCGLIB$$f3ee31b5.delete(<generated>)
de.escidoc.core.om.service.ItemHandler.delete_aroundBody4(ItemHandler.java:123)
de.escidoc.core.om.service.ItemHandler$AjcClosure5.run(ItemHandler.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
de.escidoc.core.om.service.ItemHandler.delete(ItemHandler.java:119)
de.escidoc.core.om.ejb.ItemHandlerBean.delete_aroundBody32(ItemHandlerBean.java:154)
de.escidoc.core.om.ejb.ItemHandlerBean$AjcClosure33.run(ItemHandlerBean.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.aa.security.aop.SecurityInterceptor.proceed(SecurityInterceptor.java:237)
de.escidoc.core.aa.security.aop.SecurityInterceptor.ajc$inlineAccessMethod$de_escidoc_core_aa_security_aop_SecurityInterceptor$de_escidoc_core_aa_security_aop_SecurityInterceptor$proceed(SecurityInterceptor.java:1)
de.escidoc.core.aa.security.aop.SecurityInterceptor.authorize(SecurityInterceptor.java:206)
de.escidoc.core.om.ejb.ItemHandlerBean.delete_aroundBody34(ItemHandlerBean.java:1)
de.escidoc.core.om.ejb.ItemHandlerBean$AjcClosure35.run(ItemHandlerBean.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.common.util.aop.StatisticInterceptor.proceed(StatisticInterceptor.java:205)
de.escidoc.core.common.util.aop.StatisticInterceptor.ajc$inlineAccessMethod$de_escidoc_core_common_util_aop_StatisticInterceptor$de_escidoc_core_common_util_aop_StatisticInterceptor$proceed(StatisticInterceptor.java:1)
de.escidoc.core.common.util.aop.StatisticInterceptor.createStatisticRecord(StatisticInterceptor.java:145)
de.escidoc.core.om.ejb.ItemHandlerBean.delete_aroundBody36(ItemHandlerBean.java:154)
de.escidoc.core.om.ejb.ItemHandlerBean$AjcClosure37.run(ItemHandlerBean.java:1)
org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
de.escidoc.core.om.ejb.ItemHandlerBean.delete(ItemHandlerBean.java:144)
sun.reflect.GeneratedMethodAccessor614.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.jboss.invocation.Invocation.performCall(Invocation.java:359)
org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:237)
org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:158)
org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:169)
org.jboss.ejb.plugins.CallValidationInterceptor.invoke(CallValidationInterceptor.java:63)
org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:121)
org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:350)
org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:181)
org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:168)
org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:205)
org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:138)
org.jboss.ejb.SessionContainer.internalInvoke(SessionContainer.java:648)
org.jboss.ejb.Container.invoke(Container.java:960)
sun.reflect.GeneratedMethodAccessor404.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
org.jboss.invocation.local.LocalInvoker$MBeanServerAction.invoke(LocalInvoker.java:169)
org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:118)
org.jboss.invocation.InvokerInterceptor.invokeLocal(InvokerInterceptor.java:209)
org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:195)
org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:61)
org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:70)
org.jboss.proxy.ejb.StatelessSessionInterceptor.invoke(StatelessSessionInterceptor.java:112)
org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:100)
$Proxy160.delete(Unknown Source)
sun.reflect.GeneratedMethodAccessor613.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.springframework.remoting.rmi.RmiClientInterceptorUtils.invokeRemoteMethod(RmiClientInterceptorUtils.java:111)
org.springframework.ejb.access.SimpleRemoteSlsbInvokerInterceptor.doInvoke(SimpleRemoteSlsbInvokerInterceptor.java:99)
org.springframework.ejb.access.AbstractRemoteSlsbInvokerInterceptor.invokeInContext(AbstractRemoteSlsbInvokerInterceptor.java:141)
org.springframework.ejb.access.AbstractSlsbInvokerInterceptor.invoke(AbstractSlsbInvokerInterceptor.java:189)
de.escidoc.core.common.util.service.RemoteStatelessEjbProxyFactoryBean.invoke_aroundBody2(RemoteStatelessEjbProxyFactoryBean.java:171)
de.escidoc.core.common.util.service.RemoteStatelessEjbProxyFactoryBean.invoke_aroundBody3$advice(RemoteStatelessEjbProxyFactoryBean.java:57)
de.escidoc.core.common.util.service.RemoteStatelessEjbProxyFactoryBean.invoke(RemoteStatelessEjbProxyFactoryBean.java:1)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
$Proxy253.delete(Unknown Source)
sun.reflect.GeneratedMethodAccessor612.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
java.lang.Thread.run(Thread.java:662)
]]></p></stack-trace>
</exception>
  at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:222)

Experimente[edit]

  • fgsindex.defaultWriteLockTimeout = 3000 (msec anstelle von 100.000.000) wait time for an IndexWriter for the same directory in case of a write.lock before throwing a LockObtainFailedException
de.mpg.escidoc.services.common.exceptions.TechnicalException: <exception>
  <title><h1>500 Internal eSciDoc System Error</h1></title>
  <message><p>&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;
&lt;resultPage&gt;
&lt;error&gt;
&lt;message&gt;Update error pidOrFilename=escidoc:1602273; nested exception is:
        dk.defxws.fedoragsearch.server.errors.GenericSearchException: IndexWriter new error, creating index indexName=item_container_admin :
; nested exception is:
        org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/data/coreservice/server/default/data/index/lucene/item_container_admin/write.lock&lt;/message&gt;
&lt;/error&gt;
&lt;/resultPage&gt;
</p></message>
  <class><p>de.escidoc.core.common.exceptions.system.SystemException</p></class>
  <stack-trace><p><![CDATA[
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody8(IndexingHandler.java:502)
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody9$advice(IndexingHandler.java:57)
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing(IndexingHandler.java:1)
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody6(IndexingHandler.java:338)
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing_aroundBody7$advice(IndexingHandler.java:57)
    de.escidoc.core.common.business.indexing.IndexingHandler.doIndexing(IndexingHandler.java:1)
    de.escidoc.core.common.business.indexing.IndexingHandler.indexResource(IndexingHandler.java:285)
    de.escidoc.core.common.business.indexing.IndexingHandler.addResource(IndexingHandler.java:228)
    de.escidoc.core.common.business.indexing.IndexingHandler.resourceCreated_aroundBody0(IndexingHandler.java:153)
    de.escidoc.core.common.business.indexing.IndexingHandler.resourceCreated_aroundBody1$advice(IndexingHandler.java:57)
    de.escidoc.core.common.business.indexing.IndexingHandler.resourceCreated(IndexingHandler.java:1)
    de.escidoc.core.om.business.fedora.item.ItemResourceListener.fireItemCreated(ItemResourceListener.java:98)
    de.escidoc.core.om.business.fedora.item.FedoraItemHandler.create_aroundBody6(FedoraItemHandler.java:434)
    de.escidoc.core.om.business.fedora.item.FedoraItemHandler$AjcClosure7.run(FedoraItemHandler.java:1)
    org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
    de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
    de.escidoc.core.om.business.fedora.item.FedoraItemHandler.create(FedoraItemHandler.java:409)
    de.escidoc.core.om.business.fedora.item.FedoraItemHandler$$FastClassByCGLIB$$575498c2.invoke(<generated>)
    net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191)
    org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
    org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    de.escidoc.core.om.performance.PerformanceStatisticsInterceptor.invoke_aroundBody2(PerformanceStatisticsInterceptor.java:89)
    de.escidoc.core.om.performance.PerformanceStatisticsInterceptor$AjcClosure3.run(PerformanceStatisticsInterceptor.java:1)
    org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
    de.escidoc.core.common.util.aop.TraceInterceptor.traceMethod(TraceInterceptor.java:57)
    de.escidoc.core.om.performance.PerformanceStatisticsInterceptor.invoke(PerformanceStatisticsInterceptor.java:80)
  • project FgsLucene OperationsImpl.java ohne closeIndexReader
ERROR 2013-02-12 08:35:14,415 [http-0.0.0.0-8080-8] (RESTImpl) RESTImpl error:
dk.defxws.fedoragsearch.server.errors.GenericSearchException: IndexReader open error indexName=item_container_admin :
; nested exception is:
        java.io.FileNotFoundException: /data/coreservice/server/default/data/index/lucene/item_container_admin/_7690.prx (Too many open files)
        at dk.defxws.fgslucene.OperationsImpl.getIndexReader(OperationsImpl.java:679)
        at dk.defxws.fgslucene.OperationsImpl.updateIndex(OperationsImpl.java:265)
        at dk.defxws.fedoragsearch.server.GenericOperationsImpl.updateIndex(GenericOperationsImpl.java:356)
        at dk.defxws.fedoragsearch.server.RESTImpl.updateIndex(RESTImpl.java:280)
        at dk.defxws.fedoragsearch.server.RESTImpl.doGet(RESTImpl.java:117)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
        at java.lang.Thread.run(Thread.java:662)

Network File System

Quality Assurance

HyperText Transfer Protocol

Enhanced Scientific Documentation

Unicode Transformation Format

Retrieved from "https://colab.mpdl.mpg.de/mw010/index.php?title=PubMan_Indexing_Problems&oldid=87686"