Outsource smarter

Elasticsearch .NET (using NEST) - Part 3/3

June 01, 20204 min read



Among the many wonderful benefits of Elasticsearch is its scalability, and this is, of course, possible with the cool integrations you can do with it. In this third and last part of the tutorial series, you’ll be learning how to integrate a database and the plugins you may want to add to your ES installation.


As mentioned in Part 1 regarding Elasticsearch, it is not best practice to rely on ES as your database. So if your platform or system needs one, it is advisable that you set up a separate database for it. For the purposes of the demo project in this tutorial, you’ll set up a MySQL database.



MySQL Database


The design of the database for this demo project is DATABASE FIRST. So you’ll be implementing a reverse engineering migration.

1. To create a new table, simply add a table in the MySQL workbench and save the configured table.

2. After creating the table, simply paste the following command below in the package manager console. This would reverse engineer the migration and would auto-generate the context and models.

Scaffold-DbContext:
"server=localhost;port=3306;user=root;password=yourpassword;database=yourdatabase" MySQL.Data.EntityFrameworkCore -OutputDir Models -f



Note: Make sure you've set up your database and that it has data before you perform the migration. Check Elasticsearch .NET (using NEST) - Part 2 for this.

If you have successfully configured the database, then you will be able to add data items (person or employee) in your demo project and receive a notification indicating this.



After configuring the database, the next thing to do is install the plugins you need for the demo project.


How to install on 7.5.1 on Windows



1. To install any plugin on version 7.5.1 in a Windows machine, simply navigate to the root folder of your Elasticsearch.




2. Once you are in the Elastic folder, simply type the following:


.\bin\elasticsearch-plugin.bat install NameOfThePlugin



After executing the command, it would display that it’s downloading and it will let you know once it is done.



How to install ICU ( International Components for Unicode ) ANALYSIS



The ICU plugin is a widely used library that provides support for different Unicodes (a unique number assigned to each character in every language). The plugin integrates the Lucene ICU module into Elasticsearch and extends the default Unicode library the module has. When integrated, it provides a better output when creating an analysis of Unicode libraries and languages.


Installing ICU Analysis plugin in Windows



1. There are two ways to install this plugin.

  • Download the zip file and extract it manually to the plugin folder.

  • Install any plugin on version 7.5.1 on a Windows machine, and simply navigate to the root folder of your elasticsearch.




2. Once you are in the elastic folder, simply type the following:
.\bin\elasticsearch-plugin.bat install analysis-icu




After executing the command it would display that it’s downloading and it will let you know once it is done.




Once the ICU analysis plugin has been installed, there are two ways of testing it. One is through Kibana and the other one is through the backend of your demo project.


ICU Analysis through Kibana


1. Open your Kibana Dev Tools in http://localhost:5601 and click the Tool icon (wrench).


2. Next, copy the code below.



PUT /icu_collation_analyzer?pretty
{
"settings": {
"analysis": {
"filter": {
"german_phone_book": {
"type": "icu_collation",
"language": "de",
"country": "DE",
"variant": "@collation=phonebook"
}
},
"analyzer": {
"german_phone_book":{
"tokenizer": "keyword",
"filter": [ "german_phone_book" ]
}
}
}
},
"mappings": {
"properties": {
"title": {
"type": "text",
"fields": {
"sort": {
"type":"text",
"fielddata": true,
"analyzer": "german_phone_book
}
}
}
}
}
}

The example will set up a phonebook containing all the German names. Always set fielddata:true so it would be sortable in the 7.5 elastic version. The default setting in Kibana is false due to performance issues in large amounts of data.



Note: Delete all existing indices and nodes, or restart it after installing the plugin.



Next, set up the index and add the plugin (ICU) as an analyzer. The name of the index is icu_collation_analyzer.



3. After we have created the Index, it's time to insert data (documents) into it.

4. Sort the data using collated sort:





Check the output. The order of the docs is going to be based on the German set phonebook collation.


ICU Analysis through NEST


NEST has a built-in ICU Analysis which can be seen here:

Elasticsearch-net / src / Nest / Analysis / Plugins / ICU

It will index and create tokens based on the configuration of your index. Try adding the special character below and try to search it using the “All” filter.

Special Character --> 你好

  1. Poco - Plain old CLR object.
  2. CLR - Common Language Runtime
  3. REST - Representational State Transfer
  4. JSON - Javascript Object Notation
  5. ELK - Elasticsearch Logstash Kibana
  6. DUCET - Default Unicode Collation Element Table
  7. ICU - International Components for Unicode
  8. NFD - Normalization Form Canonical Decomposition
  9. NFC - Normalization Form Canonical Composition
  10. NFKD - Normalization Form Compatilibitity Decomposition
  11. NFKC - Normalization Form Compatibility Composition



If you have gotten to this part, then I congratulate you on learning the basics of Elasticsearch. Hope this tutorial helps you on your future projects. But if you want to review everything from the beginning, kindly refer to the Parts 1 & 2 of this tutorial.

Rafael Diaz

Rafael Diaz

.NET architect

Rafael Diaz

.NET architect

Rafael (Rafa to close friends) can get quite engrossed in the tech world and especially likes reading about innovations in containerization, .NET technology (of course), microservices architecture, and Python. Surprisingly though, he can be pretty handy in the kitchen and can actually cook you some yummy paella if you’re nice and say, “Por favor!” To occupy his time when he’s not at work, he goes scuba diving, plays music or soccer, or adds snapshots to his collection of wildlife photography.

Need a better tech hire option?

Let’s connect