Elasticsearch .NET (using NEST) - Part 2/3

Views: 266

In the previous article of the series, we discussed the initial installation of Elasticsearch as well as how to boot it up successfully. Now, let’s proceed with testing it out. In case you missed the previous tutorial about installing Elasticsearch and Kibana, you can click on the chapter titles below.

Table of Contents

Part 1: Elasticsearch Basic Setup

Part 2: Elasticsearch Demo

Part 3: Elasticsearch Database and Plugins



Chapter Four: Demo Project Configuration

As with most cases in software development (or life in general, for that matter), it’s always a good idea to test out what you just installed or implemented. And that is exactly what you’re going to do here.

For this sample project, you’ll be using a Happy Path scenario/setup. You’re also going to use the following technologies:


Make sure these programs are installed and working properly before you go any further.

Using The Sample Project (.NetCore 2.1)

Another system check before you get to the good part... make sure first if you have all the following Nuget and Npm packages.

Nuget Packages:

  • NEST
  • NST.JsonNetSerializer
  • MySql.Data
  • MySql.Data.EntityFrameworkCore
  • MySql.Data.EntityFrameworkCore.Design
  • Newtonsoft.Json
  • Autofac
  • Autofac.Extensions.DependencyInjection
  • AutoMapper.Extensions.Microsoft.DepedencyInjection

Npm Packages (front-end):

  • Angular Datatables
  • Database.net
  • Hammerjs
  • Ngxspinner
  • Ngxtoastr
  • Jquery
  • AngularMaterial


The following steps are to help you make sure your MySQL server is running and that your demo application is connected with the database server.

  1. Import the schema located in the SQL folder of the application to your MySQL server.

    Note: In the base folder of the application, there is an SQL folder which has data dumps in it. It contains the schema for the database (tables and its relationships).

    a. Open your MySQL workbench (tool which is used for navigating the database) and click Server in the upper left corner. Next, click Import in the options list.

    b. At the next window, locate and select where your SQL files are located. For this demo project, it's most likely at EmployeeRecordKeeping/EmployeeRecordKeeping/SQL.

    c. Click start to import the SQL files. When done, click on the refresh icon located under the Tools tab at the upper left part. If the SQL files were successfully imported, it would show you that the database rks has been created and the tables and data are already inside.

  2. Next, for the appsettings.json, change the connection string to your configured settings.
    After importing the schema and changing the connection string, you can now run your application.
    See below for the sample.



Chapter Five: Demo Project Functionalities

So you’ve configured your demo project. Then, let’s proceed to its functionalities. First things first though, let’s briefly define the functionalities you’ll be implementing and testing for this sample project.

Create

When you create a record, what happens is that the data from the frontend will be passed through the API endpoint called https://localhost:5001/api/Employee/. This request method is called POST.

Note: It is against best practices to add a verb in the API endpoint.

From the frontend, you can start inputting data such as name, age, or address. Data will then be passed to the backend via endpoint and will go to EmployeeService. Data is then saved in the database. If successful, the method MapAndIndexEmployee() maps the data from the database and creates an index. Another method is CreateIndex() which handles the creation of the index. After we create or insert the index, we then check if the request was valid. It would return an error message if the indexing failed.

Search

In our application, we have 4 types of searches.

  • All
  • Smart Search
  • Highlight match keywords
  • Must not match

All of these requests come into a single endpoint namely:
https://localhost:5001/api/Employee/.

The method EmployeeElasticQueries() handles all of the requests and filters configured by the user. And by using NEST, we create a shortcode instead of a long JSON. After retrieving the data, we transmute the records into models.

Smart Search

Smart Search is a more advanced method of using Elasticsearch. Unlike the simple Search functionality, you will use analyzers for this.

To test the Smart Search functionality, first, you need to tell Elasticsearch that words can have different meanings. For instance, words can have synonyms. Take the Filipino word kalye; it means street in English. In order to pull up the word street when you use kalye as the search keyword, you need to open Kibana, use Dev Tools, and create an analyzer that will signal the program to perform this action as a response to the query. Before we proceed with creating indices, however, we need to truncate a schema in the database if there is any data and delete existing indexes. Follow the steps below to create or delete a custom analyzer.

How to create or delete a custom analyzer in Elasticsearch

As a general rule, use query clauses for full-text search or for any condition that can affect the relevance score. And use filters for everything else. To simplify, a filter is written in binary, which indicates two options such as if it matches the query or not, or does it have this word or not.

  1. Open Kibana and click on Dev Tools.
    Note: Access the Kibana dashboard here: http://localhost:5601/

    Below are some helpful and important commands you may need in Elasticsearch.

    C O M M A N D S A C T I O N S
    DELETE nameofyourindex Deletes your index
    GET /_cat/indices Displays the list of all the index
    POST /nameofyourindex/_open Opens the index
    POST /nameofyourindex/_close Closes the index
    GET /nameofyourindex/_search Displays all the records of your index
    GET /nameofyourindex/_mapping     Displays the mapping of your index
    (table properties if in the database)
    GET /nameofyourindex/_settings Your configuration and settings of your index


  1. Once inside Dev Tools, delete the sample script in the editor.

  2. Create the index and add an analyzer by following the json script below.
    e.g., PUT/[nameofyourindex]

    PUT /employee
    {
     "settings": {
     "analysis": {
      "filter": {
       "my_synonym_filter": {
        "type": "synonym",
        "synonyms": [
         "street,kalye"
         ]
        }
       },
       "analyzer": {
        "my_synonyms": {
         "tokenizer": "standard",
         "filter": [
           "lowercase",
           "my_synonym_filter"
           ]
          }
         }
        }
       }
      }
  3. Check the settings of your index if the analyzer has been added. If it has, then when you select Smart Search in the front end, it should work.

Congratulations, you have configured your demo project and are now able to implement and execute some functionalities.


SQL VS ELASTICSEARCH

On a side note, here are some comparisons you might want to take note of between SQL (Structured Query Language) and Elasticsearch.

S Q L E L A S T I C S E A R C H
OR SHOULD
AND MUST
NOT MUSTNOT
SCHEMA MAPPINGS
TABLES TYPES (Deprecated for version >7.X)    
COLUMNS       PROPERTY


Up next will be an explanation on the database and a tutorial on how to install the plugins. Proceed to Part 3 for the final section of this tutorial.

About the author

Rafael (Rafa to close friends) is a .NET architect at Arcanys. He can get quite engrossed in the tech world and especially likes reading about innovations in containerization, .NET technology (of course), microservices architecture, and Python. Surprisingly though, he can be pretty handy in the kitchen and can actually cook you some yummy paella if you’re nice and say, “Por favor!” To occupy his time when he’s not at work, he goes scuba diving, plays music or soccer, or adds snapshots to his collection of wildlife photography.

Be part of our growing community.

Join us and outsource smarter.

Please verify that you are not a robot.