Hi All,
Thank you for the very valuable suggestions.
I am planning to try using the Master - Slave configuration.
Best Rgds,
Mark.
On Mon, Mar 8, 2010 at 11:17 AM, Mark Miller <markrmiller@gmail.com> wrote:
On 03/08/2010 10:53 AM, Mark Fletcher wrote:
Hi Shalin,
Thank you for the mail.
My main purpose of having 2 identical cores
COREX - always serves user request
COREY - every day once, takes the updates/latest data and passess it on to
COREX.
is:-
Suppose say I have only one COREY and suppose a request comes to COREY
while
the update of the latest data is happening on to it. Wouldn't it degrade
performance of the requests at that point of time?
Yes - but your not going to help anything by using two indexes - best you
can do it use two boxes. 2 indexes on the same box will actually
be worse than one if they are identical and you are swapping between them.
Writes on an index will not affect reads in the way you are thinking - only
in that its uses IO and CPU that the read process cant. Thats going to
happen with 2 indexes on the same box too - except now you have way more
data to cache and flip between, and you can't take any advantage of things
just being written possibly being in the cache for reads.
Lucene indexes use a write once strategy - when writing new segments, you
are not touching the segments being read from. Lucene is already doing the
index juggling for you at the segment level.
So I was planning to keep COREX and COREY always identical. Once COREY has
the latest it should somehow sync with COREX so that COREX also now has
the
latest. COREY keeps on getting the updates at a particular time of day and
it will again pass it on to COREX. This process continues everyday.
What is the best possible way to implement this?
Thanks,
Mark.
On Mon, Mar 8, 2010 at 9:53 AM, Shalin Shekhar Mangar<
shalinmangar@gmail.com> wrote:
Hi Mark,
On Mon, Mar 8, 2010 at 7:38 PM, Mark Fletcher<
mark.fletcher2004@gmail.com> wrote:
I ran the SWAP command. Now:-
COREX has the dataDir pointing to the updated dataDir of COREY. So COREX
has the latest.
Again, COREY (on which the update regularly runs) is pointing to the old
index of COREX. So this now doesnt have the most updated index.
Now shouldn't I update the index of COREY (now pointing to the old
COREX)
so that it has the latest footprint as in COREX (having the latest COREY
index)so that when the update again happens to COREY, it has the latest
and
I again do the SWAP.
Is a physical copying of the index named COREY (the latest and now
datDir
of COREX after SWAP) to the index COREX (now the dataDir of COREY.. the
orginal non-updated index of COREX) the best way for this or is there
any
other better option.
Once again, later when COREY is again updated with the latest, I will
run
the SWAP again and it will be fine with COREX again pointing to its
original
dataDir (now the updated one).So every even SWAP command run will point
COREX back to its original dataDir. (same case with COREY).
My only concern is after the SWAP is done, updating the old index (which
was serving previously and now replaced by the new index). What is the
best
way to do that? Physically copy the latest index to the old one and make
it
in sync with the latest one so that by the time it is to get the latest
updates it has the latest in it so that the new ones can be added to
this
and it becomes the latest and is again swapped?
Perhaps it is best if we take a step back and understand why you need two
identical cores?
--
Regards,
Shalin Shekhar Mangar.
--
- Mark
http://www.lucidimagination.com
