INDIVIRTUAL - TECHNISCH PARTNER IN DIGITALE DIENSTVERLENING

Indexing and searching custom objects with Episerver Find and Unified Search in a multilingual site

April 25, 2019

Indexing and searching custom objects with Episerver Find and Unified Search in a multilingual site

When searching using Episerver Find you can either use Typed Search or Unified Search. Typed Search is useful if your search is targeting a specific class of objects, e.g. a specific page type. Unified Search makes it possible to search over a range of object types.

Unified Search can be a big help if you need to implement a site-wide search over all your different Episerver page types. But what if part of the content you want to search comes from an external data source? How can you return this content as part of your site-wide Unified Search search results, together with the regular Episerver pages? And how can you make sure that the custom content is returned in multiple languages, depending on the language of the search page?

When implementing these requirements I ran into some unexpected problems. The way these were solved is described in this blog post, and also implemented in a small demo project (based on the Episerver Alloy demo site).

The example in the demo project is a bit contrived, but very simple: there is a single source of widgets (accessed through a widget repository), and the widgets have both an English and a Dutch description, in different properties of the same object. When searching from the English search page, the search results should show the English description, and link to a page in the English site, where the widget will be shown with the English description. When searching from the Dutch search page, the search results should show the Dutch description, and link to a page in the Dutch site, where the widget will be shown with the Dutch description.

The basic steps which you need to take are:

  1. Create a model class for you custom content (Widget), representing the objects as retrieved from original source data.
  2. Create a model class with additional properties to make the widget data suitable for a site-wide, multilingual search (WidgetSearchResultItem).
  3. Push the WidgetSearchResultItem objects to the Episerver Find Index.
  4. Implement a method which executes the Unified Search, and returns the WidgetSearchResultItems

These basic steps, as well as the set-up of the demo project are outlined below.

General documentation on UnifiedSearch can be found here:
https://world.episerver.com/documentation/developer-guides/find/NET-Client-API/searching/Unified-search/

Installing the demo project

Pull the source of the demo project from https://github.com/johnligt/FindCustomObjects and open the project in Visual Studio.

If you have problems with missing references, please run Update-Package -reinstall from the Nuget Package Manager console.

To get the database up-and-running please run the following commands from the Nuget Package Manager console:

  1. Update-Database

  2. initialize-epidatabase

  3. Update-EPiDatabase

If for some reason you want to run an other different version of the demo project in a different folder with a different database, please change the name of the database in the connection string in the web.config, and run these three commands again.

To be able to index and search in the demo project you will need to obtain an Episerver Find developer index, through https://find.episerver.com/ and adapt the service url and the default index in the web.config with the data of your demo index.

<episerver.find serviceUrl="https://es-eu-dev-api01.episerver.net/N3rvpP0JertUUZbGp8ZmOT7Wb3Neo/" 
                defaultIndex="john.doe_epi2019johnd" />

During development and testing clear the index if necessary, through the Episerver Find admin interface in the Episerver backend (Find > Configure > Index).

After you have done all this check the enabled languages in the Episerver backend ( Admin > Config > Manage Website Languages ), and make sure only English and Nederlands (Dutch) are enabled, otherwise the language switch will not work correctly.

Create an object type for you custom content

We are dealing with custom content, in other words, content which is not a regular Episerver page. This may be content which you get from an external database, a custom table in the Episerver database, content from your custom web crawler, or content which you retrieve through a web service call. But whatever the source of your custom content, we need to represent this content in a model class.

Our model class could look something like this:

public class Widget
{ 
  public int Id { get; set; }
  public string WidgetName { get; set; }
  public string WidgetDescriptionNl { get; set; }
  public string WidgetDescriptionEn { get; set; }
}

This class represents the objects in the format in which they are retrieved from the external datasource. In the demo project these objects are provided by the WidgetRepository.

We could push these objects straight to the Episerver Find index, and you would be able to find them with a typed search. But to be able to fulfil our requirements (site-wide search in a multilingual site) we need some additional properties, for which we will set up a separate search result item class.

The WidgetSearchResultItem class

Now when setting up this class there are some things which need special attention:

You want to be able to find your custom content together with the regular EpiServer content through Unified Search. To make this possible, one option is to have the search result model class implement EPiServer.Find.UnifiedSearch.ISearchContent. The other option is to register the search result model class in the UnifiedSearchRegistry, and adding some properties with the same names as those in ISearchContent in the search result model class. The names of these properties start with “Search”. This is what is done in the demo project.

[InitializableModule]
[ModuleDependency(typeof(EPiServer.Web.InitializationModule))]
public class FindInitializationModule : IInitializableModule
{
  public void Initialize(InitializationEngine context)
  {
    var client = SearchClient.Instance;
    client.Conventions.UnifiedSearchRegistry
                .Add<WidgetSearchResultItem>();            
  }

  public void Uninitialize(InitializationEngine context) 
  {
  }
}

Another thing, which I found out the hard way: if you want to be able to filter the search results by language, your search result model class needs to implement EPiServer.Core.ILocalizable

CultureInfo ILocale.Language
{
  get => MasterLanguage;
  set => MasterLanguage = value;
}

public IEnumerable<CultureInfo> ExistingLanguages { get; set; }
public CultureInfo MasterLanguage { get; set; }

And, to be able to retrieve your custom content as part of the content of your site, your objects must have a SiteId property, which needs to be set to the ID of your site. You can see this ID in the Episerver Admin interface, in the section where you configure the websites, between the website name and website url.

editwebsite

In older Episerver instances you need to click on the website name, and then you will see the site id in the URL.

The site id is retrieved in code in the WidgetSearchResultItemService class:

var site = _siteDefinitionRepository.Get("FindCustomObjects");
var siteId = site?.Id.ToString();

The search result item class could then look something like this:

public class WidgetSearchResultItem : ILocalizable
{
  [Id]
  public string Id { get; set; }
  public string WidgetName { get; set; }
  public string WidgetDescription { get; set; }
  public string SiteId { get; set; }
  public DateTime DateIndexed { get; set; }
  public string SearchTitle { get; set; }
  public string SearchHitUrl => $"/{MasterLanguage.TwoLetterISOLanguageName}/widget/?widgetid={Id}";
  public string SearchSection { get; set; }
  public IEnumerable<string> SearchCategories { get; set; }
  public string SearchText => $"{WidgetName} {WidgetDescription}";
  public string SearchTypeName => nameof(WidgetSearchResultItem);
  public string SearchHitTypeName => nameof(WidgetSearchResultItem);

  CultureInfo ILocale.Language
  {
    get => MasterLanguage;
    set => MasterLanguage = value;
  }

  public IEnumerable<CultureInfo> ExistingLanguages { get; set; }
  public CultureInfo MasterLanguage { get; set; }        
}

Push the objects to the Episerver Find Index

Pushing the objects to the Episerver Find index is done in an Episerver scheduled job, the WidgetIndexingJob. After obtaining the list of widgets from the WidgetRepository, we call the WidgetSearchResultItemService to convert the widgets to a format which is suitable for pushing to the Episerver Find index, i.e. a list of WidgetSearchResultItem.

The salient points in the conversion are:

  1. Every WidgetSearchResultItem needs to have a unique id for Episerver Find to recognize the items in case of updates etc. The id is marked by the [ID] attribute.
  2. The SiteId property of the WidgetSearchResultItem is filled with the actual id of the site, so the content is seen as part of the website.
  3. For every widget both a Dutch and an English WidgetSearchResultItem is generated, each with a unique id.
  4. The MasterLanguage property is filled with a CultureInfo object, either for Dutch or English. This enables filtering by language.
  5. The Description field of the WidgetSearchResultItem is filled with the Dutch or the English description, depending on the language of the item.
  6. The SearchText property returns both the name and the description of the item.

The actual pushing of the objects to the index is as simple as calling

_client.Index(itemsToPush);

To prevent performance problems it is important to push the items in batches, to limit the number of calls to Episerver Find.

This is the main code in the Execute method of the scheduled job:

const int NumberOfItemsInBulkIndexAction = 10;

// Get our widgets from the datasource, through the WidgetRepository
var widgets = _widgetRepository.GetWidgets(102);

// Convert them to a format which will give the required 
// results with a multilingual Unified Search.
var listOfItemsToPush = _widgetSearchResultItemService.GetListToPushToTheIndex(widgets);

var count = listOfItemsToPush.Count;
var i = 0;

while (i <= count)
{
  var itemsToPush = listOfItemsToPush.Skip(i).Take(NumberOfItemsInBulkIndexAction);

  // Always push a list of items, never push them one by one, to
  // limit the number of calls to Episerver Find and improve performance.
  _client.Index(itemsToPush);

  i += NumberOfItemsInBulkIndexAction;  
}

Retrieving the widgets through a search page

The demo site was set up as a multilingual site, in English and Dutch, with English as the default language. A new page type was added to the Alloy site, to display the widget data. The start page, search page and widget page were translated, and a language switch was added to the page header, to facilitate testing and demo-ing.

The actual search code in the Alloy project was largely unaltered. By default UnifiedSearch will query the language branch associated with the current user. When searching from the Dutch search page, Dutch search results are returned, and when searching from the English search page, English search results are returned. By choosing a value in the “analyzer” field on the search page you can specify the language in which you want to search, whatever the language branch you are in.

The language switch

To get the required data for the language switch a method was added to the PageViewModel class: SetOtherLanguage. This method sets some additional properties on the PageViewModel, OtherLanguageUrl and OtherLanguageAbbreviation, which are sufficient to implement our simple language switch. The method is called from the constructor of PageViewModel.

public static void SetOtherLanguage<T>(PageViewModel<T> model, T page) where T : SitePageData
{
  var languageRepository = ServiceLocator.Current.GetInstance<ILanguageBranchRepository>();
  var urlResolver = ServiceLocator.Current.GetInstance<IUrlResolver>();

  var language = languageRepository.ListEnabled()
    .Where(l => !Equals(l.Culture, ContentLanguage.PreferredCulture))
    .Select(l => new LanguageSelector(l.LanguageID))
    .FirstOrDefault();

  if (language != null)
  {
    model.OtherLanguageUrl = urlResolver.GetUrl(page.ContentLink, language.Language.Name);
    model.OtherLanguageAbbreviation = language.Language.TwoLetterISOLanguageName;
    }
  }

Testing with the demo project

To see all this in action, run the project. First you will need to fill the Episerver Find Index with the widgets and the regular Episerver content. Go to the Episerver Admin interface and manually start the relevant scheduled jobs, the “Episerver Find Content Indexing Job” and the “Episerver Find Widget Indexing Job”.

You can then check the contents of the index by going to Find > Overview , where the content of the index is listed.

If you open the search page in the site and do a search on the word “widget” you should see hits in the language of the search page.

results-english

By using the language switch on the search page you should get the same search results, but with the search result description in the alternate language.

results-dutch

John Ligtenberg

John Ligtenberg

Senior .NET Developer