0

I have been asked to create a filtering functionality for e-commerce application written in PHP. Since the app is using MySQL database, I made the decision to use Elasticsearch with Logstash and Kibana.

Now this part is clear, using Logstash to transfer data from MySQL to Elasticsearch and then using Elasticsearch php client to access it.

What I'm thinking right now is if I should get the product IDs based on facet filtering results (ES aggregations) and load them from MySQL database, build the objects in PHP and fetch the results... or some other way would be good?

Also, I think the biggest challenge for me is the aggregation combinations, e. g. http://www.aliexpress.com/category/200003482/dresses.html?site=glo&shipCountry=cz&g=y&needQuery=n&pvId=326-200001275&isrefine=y where you can see that ticking another attribute will show 5 more products etc.

How to accomplish that? Should I just "pre-load" all the other possible aggregations for the next step? Cause that seems so wrong.

2

It's quite unclear what exactly you are asking. We can't write an Elasticsearch tutorial here.

As a rough overview:

You should have an ES index that contains all the information you possibly want to search/aggregate/facet and also all or most information you want to display.

Your search has two basic parts:

The search itself that returns the products. That's a combination of text search and search by terms (for the attributes like size etc).

The aggregations on facets that will return the product options that are 'left' after applying the search. That's what you display as filters.

So for your example, the only thing already selected is the 'category attribute'. This limits the search results. So following that your facets will only return attributes that are related to 'dresses' like size, color etc.

Beware that those filter attributes can either be a direct result of the search (only 'what is left') or can further filtered by the code (we don't want to show option X for clothing. (Or you can make sure that during product input it is already made impossible to add certain attributes to products in those categories).

After you got your results you can load additional info from MySQL if you have too. But it will be more performant and easier to handle for many cases if you store all info in ES where possible.

Ok, some practical code (reduced as far as possible, so removed highlighting etc):

The top level query builder:

  query: {filtered: {query: Elastic::Element.match(text)}},
  aggs: Elastic::Aggs.new(self),
  post_filter: Elastic::Element.post_filter(filters),

That's what we sent at ES. The query part, the aggs (we need to get the categories/attributes...) and a post filter. (also highlight, sort and pagination, but we ignore this here). Also it's a search query, not a count query (so we get items back, not only aggregate counts)

The query is build by a method and will look like this:

{ multi_match: {
  query:                'some text the user entered (or empty),
  type:                 'best_fields',
  operator:             'and',
  minimum_should_match: '3<75%',
  fields:               ['search_id^10', 'ean^10', 'title^1', 'text', 'text_*'],
  tie_breaker:          0.3
      }
    }

That's basically a text search here. If the user didn't enter a search term it's replaced by:

{ match_all: {} }

The post_filter is where the attributes come in:

{ bool: 
  { must: [
    {
      terms: {
        'color' => ['black', 'purple'],
        execution: 'bool',
      }
    },
    {
      terms: {
        'category' => ['dresses'],
        execution: 'bool',
      }
    }
  ]
} }

So basically the bool filter is where you add this functionality. (The terms filter is only one, though the one you will use most of the time).

So that's basically how we do it. The slightly weird syntax is because I copy/pasted this from some actual Ruby code (You would use JSON that look basically the same as far as the structuring is relevant).

Also you will use tons of helper functions, for example the whole post filter starts as an array like [{'color' => ['black', 'purple']}, 'category' => ['dresses']] which in turn is generated from the users form input.

Then this is piped through a function that generates the terms (The call to Elastic::Element.post_filter(filters) on the first level).

  • I wouldn't even want of you to write me ES tutorial :) Most of the data should be in ElasticSearch so I can use them in PHP without the need to use MySQL. What I still don't get is how to visualize functionality like MySQL IN. Lets say we have 50 dresses of various colours. I have the attribute colour and I tick "Black", after the request is processed I want to display numbers next to other colors like Purple(+6) meaning that if I perform aggregation query Colour IN ("Black","Purple") 6 new products will appear. This is what I am looking for and don't know how to do. – falnyr Jun 3 '16 at 10:54
  • @falnyr Updated, maybe that makes it a bit more clear. Elasticsearch isn't easy at first. But at one point it will make click and you will get it. From there it becomes a incredibly powerful tool. – thorsten müller Jun 3 '16 at 11:27
  • Also you should maybe use Kibana and Sense to explore search options. Create an index with like 25 articles, a few colors, categories etc so you always know what should be the correct result. The start exploring the filters as they are described in the ES docs. – thorsten müller Jun 3 '16 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.