java-user@lucene.apache.org
[Top] [All Lists]

Re: Tomcat Threads are BLOCKED after some time

Subject: Re: Tomcat Threads are BLOCKED after some time
From: damu_verse
Date: Fri, 6 Mar 2009 02:43:45 -0800 PST
Hi Yonik

            We have tried with the Lucene-2.4.0 also (JVM not changed) ..
But still threads are blocking..Not able to find the root cause...

thanks & regards
-damu

Yonik Seeley-2 wrote:
> 
> Hmmm, if this is some sort of deadlock, we may need a thread dump of
> all of the threads.
> Does your app with Lucene 2.4 have the same problem?
> Is it possible to try a newer JVM version (like 1.6.0_12)?
> 
> -Yonik
> http://www.lucidimagination.com
> 
> 
> On Tue, Mar 3, 2009 at 8:13 AM, damu_verse <damoder.reddy@xxxxxxxxx>
> wrote:
>>
>> Hi All,
>>
>> Â Â Â Â We Have used Lucene as our Search Engine and all our applications
>> are deployed onto tomcat and running with thread pool size of 200.
>>
>> Java Version - 1.6.0-rc
>> Lucene Version - 2.3.2
>> Tomcat Version - 6.0.14
>> OS - Red Hat Enterprise Linux ES release 4 (Nahant Update 5)
>> kernel - 2.6.9-55.0.2.ELsmp
>> RAM - 4 GB
>> Tomcat Memory - 1.5 GB
>> Index Size - Â2 GB
>>
>>
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â After 10-12 hrs of tomcat running, tomcat
>> becomes irresponsive. After doing core dump of tomcat process We observed
>> that all tomcat threads are blocked (Thread-pool size-200). none of the
>> tomcat threads are in runnable state.
>>
>> each thread at the time of the core dump is in BLOCKED state...The
>> following
>> are the stack trace of blocked.
>>
>> "MultiSearcher thread #3" daemon prio=10 tid=0x337ddc00 nid=0x4827
>> waiting
>> for monitor entry [0x2f2f0000..0x2f2f0ea0]
>> Â java.lang.Thread.State: BLOCKED (on object monitor)
>> Â Â Â Âat
>> org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:235)
>> Â Â Â Â- waiting to lock <0x45d49d88> (a
>> org.apache.lucene.store.FSDirectory$FSIndexInput)
>> Â Â Â Âat
>> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:152)
>> Â Â Â Âat
>> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
>> Â Â Â Âat org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:76)
>> Â Â Â Âat org.apache.lucene.index.TermBuffer.read(TermBuffer.java:63)
>> Â Â Â Âat
>> org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:123)
>> Â Â Â Âat
>> org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:154)
>> Â Â Â Âat
>> org.apache.lucene.index.TermInfosReader.scanEnum(TermInfosReader.java:223)
>> Â Â Â Âat
>> org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:217)
>> Â Â Â Âat
>> org.apache.lucene.index.SegmentTermDocs.seek(SegmentTermDocs.java:54)
>> Â Â Â Âat
>> org.apache.lucene.index.IndexReader.termDocs(IndexReader.java:668)
>> Â Â Â Âat
>> org.apache.lucene.search.ConstantScoreTermQuery$TermWeight.scorer(ConstantScoreTermQuery.java:63)
>> Â Â Â Âat
>> org.apache.lucene.search.VBooleanQuery$BooleanWeight.scorer(VBooleanQuery.java:276)
>> Â Â Â Âat
>> org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:232)
>> Â Â Â Âat
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:143)
>> Â Â Â Âat
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:124)
>> Â Â Â Âat
>> org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.java:250)
>>
>>
>>
>> "http-8080-194" daemon prio=10 tid=0x08927800 nid=0x128d waiting for
>> monitor
>> entry [0x2e188000..0x2e189e20]
>> Â java.lang.Thread.State: BLOCKED (on object monitor)
>> Â Â Â Âat
>> org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:235)
>> Â Â Â Â- waiting to lock <0x45d49d88> (a
>> org.apache.lucene.store.FSDirectory$FSIndexInput)
>> Â Â Â Âat
>> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:152)
>> Â Â Â Âat
>> org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
>> Â Â Â Âat
>> org.apache.lucene.store.IndexInput.readVLong(IndexInput.java:96)
>> Â Â Â Âat
>> org.apache.lucene.index.MultiLevelSkipListReader.loadSkipLevels(MultiLevelSkipListReader.java:196)
>> Â Â Â Âat
>> org.apache.lucene.index.MultiLevelSkipListReader.skipTo(MultiLevelSkipListReader.java:97)
>> Â Â Â Âat
>> org.apache.lucene.index.SegmentTermDocs.skipTo(SegmentTermDocs.java:164)
>> Â Â Â Âat in.verse.search.query.spans.TermSpans.skipTo(TermSpans.java:85)
>> Â Â Â Âat
>> in.verse.search.query.spans.SpanScorer.skipTo(SpanScorer.java:70)
>> Â Â Â Âat
>> org.apache.lucene.search.VConjunctionScorer.doNext(VConjunctionScorer.java:78)
>> Â Â Â Âat
>> org.apache.lucene.search.VConjunctionScorer.next(VConjunctionScorer.java:71)
>> Â Â Â Âat
>> org.apache.lucene.search.VBooleanScorer2.next(VBooleanScorer2.java:456)
>> Â Â Â Âat
>> org.apache.lucene.search.VConjunctionScorer.init(VConjunctionScorer.java:136)
>> Â Â Â Âat
>> org.apache.lucene.search.VConjunctionScorer.next(VConjunctionScorer.java:65)
>> Â Â Â Âat
>> org.apache.lucene.search.VBooleanScorer2.score(VBooleanScorer2.java:412)
>> Â Â Â Âat
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)
>> Â Â Â Âat
>> org.apache.lucene.search.ParallelMultiSearcher.search(ParallelMultiSearcher.java:173)
>> Â Â Â Âat org.apache.lucene.search.Searcher.search(Searcher.java:118)
>> Â Â Â Âat org.apache.lucene.search.Searcher.search(Searcher.java:97)
>> Â Â Â Âat VSearcher$VerticalSearcher$1.call(VSearcher.java:500)
>> Â Â Â Âat VSearcher$VerticalSearcher$1.call(VSearcher.java:491)
>> Â Â Â Âat
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> Â Â Â Âat java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> Â Â Â Âat
>> java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:1737)
>> Â Â Â Âat
>> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
>> Â Â Â Âat
>> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
>> Â Â Â Âat
>> java.util.concurrent.AbstractExecutorService.invokeAll(AbstractExecutorService.java:200)
>> Â Â Â Âat
>> in.verse.search.VSearcher$VerticalSearcher.searchInternal(VSearcher.java:542)
>> Â Â Â Âat
>> in.verse.search.VSearcher$VerticalSearcher.searchInternal(VSearcher.java:476)
>> Â Â Â Âat
>> in.verse.search.VSearcher$VerticalSearcher.search(VSearcher.java:384)
>> Â Â Â Âat in.verse.search.VSearcher.search(VSearcher.java:187)
>> Â Â Â Âat
>> in.verse.search.impl.SearchQuery.executeSearchQuery(SearchQuery.java:166)
>> Â Â Â Âat
>> in.verse.search.impl.SearchQuery.executeSearchQuery(SearchQuery.java:255)
>> Â Â Â Âat
>> org.apache.jsp.jobs.mobile.airtel.xhtml.search_jsp._jspService(search_jsp.java:869)
>> Â Â Â Âat
>> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
>> Â Â Â Âat javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
>> Â Â Â Âat
>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:393)
>> Â Â Â Âat
>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:320)
>> Â Â Â Âat
>> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:266)
>> Â Â Â Âat javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
>> Â Â Â Âat
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>> Â Â Â Âat
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>> Â Â Â Âat
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>> Â Â Â Âat
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
>> Â Â Â Âat
>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
>> Â Â Â Âat
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
>> Â Â Â Âat
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>> Â Â Â Âat
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>> Â Â Â Âat
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263)
>> Â Â Â Âat
>> org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:852)
>> Â Â Â Âat
>> org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:584)
>> Â Â Â Âat
>> org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1508)
>> Â Â Â Âat java.lang.Thread.run(Thread.java:619)
>> --
>> View this message in context:
>> http://www.nabble.com/Tomcat-Threads-are-BLOCKED-after-some-time-tp22308385p22308385.html
>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@xxxxxxxxxxxxxxxxx
>> For additional commands, e-mail: java-user-help@xxxxxxxxxxxxxxxxx
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@xxxxxxxxxxxxxxxxx
> For additional commands, e-mail: java-user-help@xxxxxxxxxxxxxxxxx
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Tomcat-Threads-are-BLOCKED-after-some-time-tp22308385p22369462.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@xxxxxxxxxxxxxxxxx
For additional commands, e-mail: java-user-help@xxxxxxxxxxxxxxxxx

<Prev in Thread] Current Thread [Next in Thread>