This forum is in READ-ONLY mode.
You can look around, but if you want to ask a new question, please use Stack Overflow.

Zend Lucene and related models (with foreign keys)

Questions relating to sfLucenePlugin, sfSearch etc

Zend Lucene and related models (with foreign keys)

by petersaints » Tue Oct 05, 2010 2:41 am

Well I was developing an application when I realized a major drawback on Zend Lucene (or on the way I want to use it).

I have a model called Publication that is related with a few other models (subjects, genres, languages, authors, etc.) and I'm getting they're names when adding a new document to the index (using the Jobeet tutorial way) so that I can search for publications with a given subject, genre, language, author, etc... The problem is if for some reason I decide to alter the name of one of those related models the Zend Lucene index will not get updated.

The only two solutions I could come up with were:
1. Re-index all the publications regularly to guarantee that any changes made to the related models gets updated on the index (however this solution doesn't allow the index to be updated in real time)
2. Get all the publications that are related with a given model and re-index them after it gets updated (using the save(), postSave(), postUpdate() or whatever you can come up with on Doctrine).
--> This solution seems great... It will only rebuild the index for the publications that are linked to the updated model right? Well, if you have something like a thousand (1000) publications linked to it will take a few minutes (yeah I tested it) to update and on a user form it will timeout because it takes over 30 seconds (and even if it don't it would be bad to have a user looking at the screen for a few minutes awaiting for the page to finish to load).

So what I want to know is if there's another solution? Is there a way to update an index on the fly based on a change on a related model without hanging the whole pahe? Maybe putting the task to run on the background or something? Is there such a way?

If there's no way to do this with Lucene is there any way to use Full-Text Search with MySQL (with InnoDB tables) without using Zend Lucene that doesn't have such a drawback? If there's such a tool I'd glady refactor my code to accommodate a different library.

Could you please help me with this? Thanks in advance!
petersaints
Junior Member
 
Posts: 23
Joined: Tue Mar 09, 2010 9:19 pm
Location: Portugal

Re: Zend Lucene and related models (with foreign keys)

by petersaints » Wed Oct 06, 2010 6:15 pm

Well... I'm answering my self. After thinking about it for some time I ended with a compromisse solution.

On my model I already have a one-to-one relationship with a table that is used only for storing meta-information about a publication so I ended up inserting a new column called reindex (that is a "boolean"). This way everytime I update an entity related with a publication (something that in production will happen very seldom but I want to be prepared for it) it will mark every publication that is related to it as needing reindexing. Than I have a task that can be run on cron job or Task Scheduler that will only reindex the publications that are marked as needing it. This way I can set this task to run a few times a week at late hours to keep the Index consistent.

It's not a perfect solution but is the best I can came up with using only PHP and Zend Lucen
petersaints
Junior Member
 
Posts: 23
Joined: Tue Mar 09, 2010 9:19 pm
Location: Portugal

Re: Zend Lucene and related models (with foreign keys)

by halfer » Tue Oct 26, 2010 5:52 pm

There would probably more ZL users over in a dedicated forum someplace. You're welcome to posit this sort of query here, but there may not be many expert users :)
halfer
Faithful Member
 
Posts: 10148
Joined: Mon Jan 30, 2006 1:16 pm
Location: West Midlands, UK