lucene i18n

[lekin]

Hi!

I don't usually post on this forum since i usually find answers to my questions thanks to google and forums.

since i've been struggling to find the info on this, i thought i would share my solution.

i wanted to index my i18n movie database with lucene to use it as my search engine.
my code is based on the jobeet tutorial.

everything went fine until i discovered the search engine didn't return any result when i would search for a movie name in french.

here's what my index function used to be:

Code: Select all

 public function updateLuceneIndex()
  {
    $index = MovieTable::getLuceneIndex();
   
    // remove existing entries
    foreach ($index->find('pk:'.$this->getId()) as $hit)
    {
      $index->delete($hit->id);
    }
   

    $doc = new Zend_Search_Lucene_Document();
   
    // store movie primary key to identify it in the search results
    $doc->addField(Zend_Search_Lucene_Field::Keyword('pk', $this->getId()));
   
    // index movie fields
    $doc->addField(Zend_Search_Lucene_Field::UnStored('title', $this->getTitle(), 'utf-8'));
    $doc->addField(Zend_Search_Lucene_Field::UnStored('synopsis', $this->getSynopsis(), 'utf-8'));
    $doc->addField(Zend_Search_Lucene_Field::UnStored('director', $this->getDirector(), 'utf-8'));
    $doc->addField(Zend_Search_Lucene_Field::UnStored('casting', $this->getCasting(), 'utf-8'));
   
    // add movie to the index
    $index->addDocument($doc);
    $index->commit();
   
  }

and my table schema, as a reference:

Code: Select all

Movie:
  actAs:
    Timestampable: ~
    I18n:
      fields: [synopsis, title, presskit]
      actAs:
        Sluggable: { fields: [title], uniqueBy: [lang, title] }
  columns:
    id:
      type: integer(4)
      primary: true
      notnull: true
      autoincrement: true
    title:
      type: string(255)
    synopsis:
      type: clob(65535)
    presskit:
      type: string(255)
    director:
      type: string(255)
    screenplay:
      type: string(255)
    producers:
      type: string(255)
  relations:
    Categories:
      class: Category
      local: movie_id
      foreign: category_id
      refClass: MovieCategory
      foreignAlias: Users



so the reason why my french title and synopsis didn't get indexed was $this->getTitle() and $this->getSynopsis would only return the current culture title and synopsis,
which for any reason would default to english even though my default culture is set to french.

anyway, so i needed to find a way to force the access to my localized data.
Thanks to someone somewhere on another forum i was able to get the tip which is $this->Translation['en'].
some (few?) people seem to know about that but since it's undocumented i figured i would share it again.

so i created two i18n accessors in my Movie.class.php:

Code: Select all

  public function getTitleI18n($culture)
  {
    return $this->Translation[$culture]['title'];
  }
 
  public function getSynopsisI18n($culture)
  {
    return $this->Translation[$culture]['synopsis'];
  }


and here's my final lucene index method:

Code: Select all

  public function updateLuceneIndex()
  {
    $index = MovieTable::getLuceneIndex();
   
    // remove existing entries
    foreach ($index->find('pk:'.$this->getId()) as $hit)
    {
      $index->delete($hit->id);
    }
   

    $doc = new Zend_Search_Lucene_Document();
   
    // store movie primary key to identify it in the search results
    $doc->addField(Zend_Search_Lucene_Field::Keyword('pk', $this->getId()));
   
    // index movie fields
    $doc->addField(Zend_Search_Lucene_Field::UnStored('title_fr', $this->getTitleI18n('fr'), 'utf-8'));
    $doc->addField(Zend_Search_Lucene_Field::UnStored('title_en', $this->getTitleI18n('en'), 'utf-8'));
    $doc->addField(Zend_Search_Lucene_Field::UnStored('synopsis_fr', $this->getSynopsisI18n('fr'), 'utf-8'));
    $doc->addField(Zend_Search_Lucene_Field::UnStored('synopsis_en', $this->getSynopsisI18n('en'), 'utf-8'));
    $doc->addField(Zend_Search_Lucene_Field::UnStored('director', $this->getDirector(), 'utf-8'));
    $doc->addField(Zend_Search_Lucene_Field::UnStored('casting', $this->getCasting(), 'utf-8'));
   
    // add movie to the index
    $index->addDocument($doc);
    $index->commit();
   
  }


works like a charm!

hopefully this will help someone!

i take this opportunity to again thanks everyone involved in symfony, its creators obviously, and the whole community behind it.
since i've started using symonfy (1.0) a few years back, i've been able to deliver higher quality projects and focus on other aspects (details, user experience, etc.)

THANKS A LOT!