Learn How To Configure Elasticsearch MongoDB River

Learn how to create mongodb river in elasticsearch.All the documents of specified mongodb collection will be indexed into elasticsearch.

Prerequisites -

  • Install Elasticsearch on Ubuntu 14.04
  • Follow this link to install elasticsearch as a service.
  • Install Mongodb on Ubuntu 14.04

Please make sure your elasticsearch and mongodb version are according to the table shown in below link.

https://github.com/richardwilly98/elasticsearch-river-mongodb

 

Step 1

The elasicsearch mongodb river plugin is dependent on elasticsearch-mapper-attachment plugin.

Generally elasticsearch path in Ubuntu 14.04 is /usr/share/elasticsearch

ES_HOME = /usr/share/elasticsearch.

Use version according to the table.

ES_HOME/bin/plugin -install elasticsearch/elasticsearch-mapper-attachments/x.x.x

Step 2

After installing the above plugin,install elasticsearh mongodb river plugin

ES_HOME/bin/plugin --install com.github.richardwilly98.elasticsearch/elasticsearch-river-mongodb/2.0.4

Step 3

Also,install this plugin to keep track of indices and its data.A web front end for an Elasticsearch cluster.

ES_HOME/bin/plugin -install mobz/elasticsearch-head
open http://localhost:9200/_plugin/head/

Step 4

Enable mongodb replica set. Learn more about replication.

Edit /etc/mongodb.conf file,add this line -

replSet=rs0

Save and exit.

Step 5

Get into mongo terminal.Then type

rs.initiate()

rs.initiate()

rs.status()

rsstatus

Step 6

Create a river for mongodb collection.Open terminal and type -


curl -XPUT "localhost:9200/_river/river_name/_meta" -d '
{
"type": "mongodb",
"mongodb": {
"servers": [
{ "host": "localhost", "port": 27017 }
],
"options": { "secondary_read_preference": true },
"db": "db_name",
"collection": "collection_name"
},
"index": {
"name": "arbitary_index_name",
"type": "arbitary_index_type"
}
}'

For simplification, keep arbitary_index_type equal to river_name and arbitary_index_name equal to collection_name.You will get a index created true message in the terminal.

Step 4

Point your browser to

http://localhost:9200/_plugin/head/

Check how many documents have been indexed.
Goto - 

http://localhost:9200/arbitary_index_name/

Bingo!!!!!! Done.Post your queries in comment section.

  • Mandeep Gulati

    I followed all the steps but it does not copy the data from mongo to ES. Please see the question on stackoverflow
    http://stackoverflow.com/questions/27582003/unable-to-copy-data-from-mongodb-to-elasticsearch?noredirect=1#comment43589684_27582003

  • http://www.codetweet.com/ Jitendra Sisodiya

    great article.
    step by step illustration helped me in configuring river easily.

  • Prem Reddy

    Your article helped me setup the whole system and working fine.But If I update mongodb fields it’s not updating in elasticsearch indexes.Once it updated itself.But now It’s not updating I restarted and still no updation.I know repsets are for this purpose only.But Could you shed some light on my problem.Thank You

  • moghira

    How about doing it with AWS?