csv - How can I indicate the entry id in the Logstash script? -


i have following mapping in elasticsearch:

put /myindex {   "mappings": {     "regions": {       "_all": {         "enabled": false       },        "properties": {           "regionid": {"type": "keyword"},           "seasons": {                "properties": {                   "season-type":  { "type": "keyword" },                   "period": {                      "properties": {                       "start-day":  { "type": "integer" },                       "start-month":  { "type": "integer" },                       "end-day":  { "type": "integer" },                       "end-month":  { "type": "integer" }                     }                   }              }             },             "autonomous_communities": {                 "properties": {                       "autonomous_community":  { "type": "keyword" },                       "codified_autonomous_community":  { "type": "keyword" }                 }             }         }     }   } } 

i save input data follows:

put myindex/regions/_bulk {"index":{"_id":"1"}} {"regionid":"1", "seasons":[{"season-type":"type1","start-day":10,"start-month":1,"end-day":31,"end-month":5},{"season-type":"type2","start-day":1,"start-month":9,"end-day":9,"end-month":1},{"season-type":"type3","start-day":1,"start-month":6,"end-day":30,"end-month":9}]} 

however field autonomous_communities should filled csv file, because contains large array. so, use following logstash script add data autonomous_communities_shapefile.

input {     file {         path => ["/path/to/csv/file.csv"]         sincedb_path => "/dev/null"         start_position => beginning     } }  filter {       csv {         columns => ["autonomous_community","codified_autonomous_community"]         separator => ","     }     if [col1] == "autonomous_community" {     drop {}     } }  output {     stdout { codec => rubydebug }     elasticsearch {         action => "index"         hosts => ["127.0.0.1:9200"]         index => "myindex"         document_type => "autonomous_communities"         workers => 1     } } 

how can indicate corresponding regionid in logstash script? imagine have 10 entries regionid 1 10. so, current csv file refers regionid equal 1.

my objective see following result in elasticsearch after running logstash script:

"hits": [       {         "_index": "myindex",         "_type": "regions",         "_id": "1",         {"regionid":"1",           "seasons":[{"season-type":"type1","start-day":10,"start-month":1,"end-day":31,"end-month":5},{"season-type":"type2","start-day":1,"start-month":9,"end-day":9,"end-month":1},{"season-type":"type3","start-day":1,"start-month":6,"end-day":30,"end-month":9}],                "autonomous_communities":[{"autonomous_community":"xxx","codified_autonomous_community":"aaa"},{"autonomous_community":"yyy","codified_autonomous_community":"bbb"}]          }   } ] 

but this:

"hits": [       {         "_index": "myindex",         "_type": "regions",         "_id": "1",         {"regionid":"1",           "seasons":[{"season-type":"type1","start-day":10,"start-month":1,"end-day":31,"end-month":5},{"season-type":"type2","start-day":1,"start-month":9,"end-day":9,"end-month":1},{"season-type":"type3","start-day":1,"start-month":6,"end-day":30,"end-month":9}],         },   {         "_index": "myindex",         "_type": "regions",         "_id": "345864068",               "autonomous_community":"xxx",         "codified_autonomous_community":"aaa"   },   {         "_index": "myindex",         "_type": "regions",         "_id": "57934753670,               "autonomous_community":"yyy",         "codified_autonomous_community":"bbb"    } ] 


Comments

Popular posts from this blog

ZeroMQ on Windows, with Qt Creator -

unity3d - Unity SceneManager.LoadScene quits application -

python - Error while using APScheduler: 'NoneType' object has no attribute 'now' -