csv - How can I indicate the entry id in the Logstash script? -

-

i have following mapping in elasticsearch:

put /myindex {   "mappings": {     "regions": {       "_all": {         "enabled": false       },        "properties": {           "regionid": {"type": "keyword"},           "seasons": {                "properties": {                   "season-type":  { "type": "keyword" },                   "period": {                      "properties": {                       "start-day":  { "type": "integer" },                       "start-month":  { "type": "integer" },                       "end-day":  { "type": "integer" },                       "end-month":  { "type": "integer" }                     }                   }              }             },             "autonomous_communities": {                 "properties": {                       "autonomous_community":  { "type": "keyword" },                       "codified_autonomous_community":  { "type": "keyword" }                 }             }         }     }   } }

i save input data follows:

put myindex/regions/_bulk {"index":{"_id":"1"}} {"regionid":"1", "seasons":[{"season-type":"type1","start-day":10,"start-month":1,"end-day":31,"end-month":5},{"season-type":"type2","start-day":1,"start-month":9,"end-day":9,"end-month":1},{"season-type":"type3","start-day":1,"start-month":6,"end-day":30,"end-month":9}]}

however field autonomous_communities should filled csv file, because contains large array. so, use following logstash script add data autonomous_communities_shapefile.

input {     file {         path => ["/path/to/csv/file.csv"]         sincedb_path => "/dev/null"         start_position => beginning     } }  filter {       csv {         columns => ["autonomous_community","codified_autonomous_community"]         separator => ","     }     if [col1] == "autonomous_community" {     drop {}     } }  output {     stdout { codec => rubydebug }     elasticsearch {         action => "index"         hosts => ["127.0.0.1:9200"]         index => "myindex"         document_type => "autonomous_communities"         workers => 1     } }

how can indicate corresponding regionid in logstash script? imagine have 10 entries regionid 1 10. so, current csv file refers regionid equal 1.

my objective see following result in elasticsearch after running logstash script:

"hits": [       {         "_index": "myindex",         "_type": "regions",         "_id": "1",         {"regionid":"1",           "seasons":[{"season-type":"type1","start-day":10,"start-month":1,"end-day":31,"end-month":5},{"season-type":"type2","start-day":1,"start-month":9,"end-day":9,"end-month":1},{"season-type":"type3","start-day":1,"start-month":6,"end-day":30,"end-month":9}],                "autonomous_communities":[{"autonomous_community":"xxx","codified_autonomous_community":"aaa"},{"autonomous_community":"yyy","codified_autonomous_community":"bbb"}]          }   } ]

but this:

"hits": [       {         "_index": "myindex",         "_type": "regions",         "_id": "1",         {"regionid":"1",           "seasons":[{"season-type":"type1","start-day":10,"start-month":1,"end-day":31,"end-month":5},{"season-type":"type2","start-day":1,"start-month":9,"end-day":9,"end-month":1},{"season-type":"type3","start-day":1,"start-month":6,"end-day":30,"end-month":9}],         },   {         "_index": "myindex",         "_type": "regions",         "_id": "345864068",               "autonomous_community":"xxx",         "codified_autonomous_community":"aaa"   },   {         "_index": "myindex",         "_type": "regions",         "_id": "57934753670,               "autonomous_community":"yyy",         "codified_autonomous_community":"bbb"    } ]


Comments

Post a Comment

Popular posts from this blog

ios - MKAnnotationView layer is not of expected type: MKLayer -

-
Image
so code works fine logger riddled message. there way rid of or suppress it? postannotation.swift class postannotation: mkpointannotation { //mark: properties let post: post //mark: initialization init(post: post) { self.post = post super.init() self.coordinate = cllocationcoordinate2d(latitude: post.latitude, longitude: post.longitude) self.title = post.title self.subtitle = post.timestring() } } adding annotation let annotation = postannotation(post: post) self.map.addannotation(annotation) func mapview func mapview(_ mapview: mkmapview, viewfor annotation: mkannotation) -> mkannotationview? { if annotation mkuserlocation { return nil } var annotationview = mapview.dequeuereusableannotationview(withidentifier: "pin") as? mkpinannotationview if annotationview == nil { annotationview = mkpinannotationview(annotation: annotation, reuseidentifier: "pin")
Read more

ZeroMQ on Windows, with Qt Creator -

-
i have installed zmq link given below on windows 7. http://zeromq.org/distro:microsoft-windows 1) have c++ project in qt-creator , i'm using zmq c++ binding (zhelpers.hpp etc). 2) have added .lib installed zmq folder qtcreator 3) while compiling, compiler looks _imp__zmq_verion, in .dll files present in .lib have __imp_zmq_version. 4) causing undefined reference zmq_version. anyone else faced problem , has solution ?
Read more

unity3d - Unity SceneManager.LoadScene quits application -

-
i have 2 different scenes 1 named 'main' , other 'game' both added scenes in load. when use scenemanager.loadscene("game"); works no problem when tried return main scene code if (input.getkeydown (keycode.escape)) { scenemanager.loadscene("main"); } strangely game quits. thanks. sorry interruption. somewhere on other script found code. if (input.getkeydown (keycode.escape)) { application.quit (); } that's bad, know...
Read more