hadoop - How to parse for occurences based on inputs in the same file -

- 

event1           foo_id1 event1           foo_id2 event1           foo_id4 event1           foo_id6 event1           foo_id7 event1           foo_id8 event1           foo_id8 event1           foo_id1 event1           foo_id4  event2           foo_id1 event2           foo_id2 event2           foo_id3 event2           foo_id4 event2           foo_id5 event2           foo_id6 event2           foo_id8 event2           foo_id9 event2           foo_id11

the above information available file in s3 under bucket (say s3://hadoop.mycompany.com/bucket1/foo1.txt).

all events have foo_ids. events in "event2", know how many times foo_id(s) occur in event1.

e.g. in above case,

foo_id1=2 foo_id2=1 foo_id3=0 foo_id4=2 foo_id5=0 foo_id6=1 foo_id8=2 foo_id9=0 foo_id11=0

how write hive script return data in expected format?

hi can accomplished using following hive script:

  1. first need create hive external table using command

    create external table events (event string, foo string) row format delimited fields terminated '\t' location 's3n://hadoop.mycompany.com/bucket1/';

  2. run following query

    select e2.foo, count(e1.foo) events e2 left outer join events e1 on e1.foo = e2.foo , e1.event = 'event1' e2.event = 'event2' group e2.foo;

you should results need, this:

foo_id1  2 foo_id11 0 foo_id2  1 foo_id3  0 foo_id4  2 foo_id5  0 foo_id6  1 foo_id8  2 foo_id9  0

hope solves problem.


Comments

Post a Comment

Popular posts from this blog

Perl - how to grep a block of text from a file -

-
it can in xml or text format. how in general grep block of text in perl? <track type="ws"> <range> <rangestart>0</rangestart> <rangeend>146.912</rangeend> <locationindex>0</locationindex> <propertyindex>0</propertyindex> </range> </track> <track type="ps" id="1"> <range> <rangestart>0</rangestart> <rangeend>146.912</rangeend> <locationindex>1</locationindex> <propertyindex>1</propertyindex> </range> </track> i want grep type="ps" , till </range> . one solution open file, read line line , match block. open(fh, "file.txt"); foreach $line (<fh>) { if ($line =~ m/type="cc"(.*?)<\/range>/) { print $1; } } but there more optimal solution without reading file line l
Read more

delphi - How to remove all the grips on a coolbar if I have several coolbands? -

-
Image
if set mycoolbar.fixedorder true,only grip on first band hidden. well,if use delphi 7 create vcl forms application,then put coolbar on , create 3 coolbans hold other controls,only grip , top coolbands can hidden setting mycoolbar.fixedorder:=true. i've uploaded picture make things clear. you got fixedorder property wrong. fixed property not allow bands rearranged if set true . setting property of coolbar true keep user changing bands order @ runtime, user can still move , resize bands. can give advice can do, actual solution problem, well, have wait answer. my advice use 3 coolbars in row , setting "fixedorder" property true , bandborderstyle bsnone . way grip hidden on of them. about property, it's not bug of ide, actual preference of property.
Read more

javascript - Animating array of divs; only the final element is modified -

-
this baffling me @ moment. i'm experienced programmer looking throw bit of surprise fiancee on countdown page our wedding. desired effect simulated confetti falling behind main page. setup follows: i have array of divs in javascript. each div absolutely positioned on page. have interval set updates every 50 or milliseconds. on each tick, run loop on array of divs, , calculations , update positions. problem problem having, however, though can see divs created, stored array, , modified (in case, each has slight randomized rotation) during initialization phase, once tick update position starts, div stored in whatever last slot of array gets position updated. tried hard-coding each div index testing purposes, strictly last element update position. upon printing current "left" , "top" style values each div before , after calculations shows value has been changed, no visible change noticeable. here's basic code: javascript var container;
Read more