ojb-user@db.apache.org
[Top] [All Lists]

RE: Query problem for hundreds of thousands of records

Subject: RE: Query problem for hundreds of thousands of records
From: "Joe Latty"
Date: Wed, 13 Oct 2004 13:46:31 +1000
Without going into why you would want a query to return 350k records,
you may have to change some parameters in the OJB.properties file.

Look for the following and you may need to change maxActive to a
non-positive number. This can give out of memory errors.

# maximum number of brokers that can be borrowed from the
# pool at one time. When non-positive, there is no limit.
maxActive=100

-----Original Message-----
From: WHIRLYCOTT [mailto:phil@xxxxxxxxxxxxxx] 
Sent: Wednesday, 13 October 2004 12:02 PM
To: OJB Users List
Subject: Re: Query problem for hundreds of thousands of records

If you need the entire resultset in memory, you probably need to provide

more memory to your jvm or add more memory physically to your hardware.

It sounds like, in effect, that you are basically making a copy of your 
database table with ~350k records in memory!

What happens when 2 users hit your website at the same time?  What 
happens when 100 users hit your site?

I don't know the specifics of what you are doing, but it doesn't sound 
like a particularily good solution.  Why do you think you need to have 
the entire result set in memory?

phil.

[ Muliawan Sjarif ] wrote:

> If I need the entire resultset in the memory what would be the best
> approach? It seems that broker.getCollectionByQuery(q) stuck with huge
> records.
> 
> 
> On Tue, 12 Oct 2004 16:09:11 -0400, WHIRLYCOTT <phil@xxxxxxxxxxxxxx>
wrote:
> 
>>You probably are doing something like this:
>>
>>       Collection c = broker.getCollectionByQuery(q);
>>
>>... and ought to be doing something like this:
>>
>>       Iterator i = broker.getIteratorByQuery(q);
>>
>>Unless you need the entire result set in memory, this should be your
>>preferred approach.  Just iterate through the results and do what you
>>need to do to each of them.
>>
>>phil.
>>
>>[ Muliawan Sjarif ] wrote:
>>
>>
>>>Hi,
>>>
>>>I was doing a quick test for OJB particularly using PersistenceBroker
>>>on one table that consists of 352544 records. No matter how many
times
>>>I tried, in the end all I got was the browser not responding (no
>>>result displayed on the page) and tomcat throwed
java.lang.OutOfMemory
>>>exception. For this test, dynamic proxy was utilized.
>>>
>>>Pertaining to this problem, I wonder if anyone has encountered
similar
>>>problem. Or is there somewhere in the OJB's properties that need to
be
>>>tweaked?
>>>
>>>Many thanks.
>>>
>>>Regards,
>>>Muliawan
>>
>>-- 
>>                                  Whirlycott
>>                                  Philip Jacob
>>                                  phil@xxxxxxxxxxxxxx
>>                                  http://www.whirlycott.com/phil/
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: ojb-user-unsubscribe@xxxxxxxxxxxxx
>>For additional commands, e-mail: ojb-user-help@xxxxxxxxxxxxx
>>
>>
> 
> 
> 

-- 
                                   Whirlycott
                                   Philip Jacob
                                   phil@xxxxxxxxxxxxxx
                                   http://www.whirlycott.com/phil/

---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@xxxxxxxxxxxxx
For additional commands, e-mail: ojb-user-help@xxxxxxxxxxxxx


---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@xxxxxxxxxxxxx
For additional commands, e-mail: ojb-user-help@xxxxxxxxxxxxx

<Prev in Thread] Current Thread [Next in Thread>