web scraping - Scrape a Google Chart script with Scraperwiki (Python) -

-

i'm getting scraping scraperwiki in python. figured out how scrape tables page, run scraper every month , save results on top of each other. pretty cool.

now want scrape page information on android versions , run script monthly. in particular, want table version, codename, api , distribution. it's not easy.

the table called wrapper div. there way scrape information? can't find solution.

plan b scrape visualisation. need, codename , percentage, that's sufficient. information can found in html in google chart script.

google chart api script

but can't find information 'souped' html. have a public scraper on here. can edit make work.

can explain how can approach problem? working scraper comments on what's going on awesome.

this difficult case, because kisamoto mentioned, data inside embedded javascript , not in seperate json file expect. possible beautifulsoup involes ugly string processing: 

last_paragraph = soup.find_all('p', style='clear:both')[-1] script_tag = last_paragraph.next_sibling.next_sibling script_text = script_tag.text  lines = script_text.split('\n') data_text = '' line in lines:      if 'screen_data' in line: break     data_text = data_text + line  data_text = data_text.replace('var version_data =', '') # delete semicolon @ end data_text = data_text[:-1]  data = json.loads(data_text) data = data[0] print data['data']

output: 

[{u'perc': u'0.1', u'api': 4, u'name': u'donut'}, ... ]

Comments

Post a Comment

Popular posts from this blog

Perl - how to grep a block of text from a file -

-
it can in xml or text format. how in general grep block of text in perl? <track type="ws"> <range> <rangestart>0</rangestart> <rangeend>146.912</rangeend> <locationindex>0</locationindex> <propertyindex>0</propertyindex> </range> </track> <track type="ps" id="1"> <range> <rangestart>0</rangestart> <rangeend>146.912</rangeend> <locationindex>1</locationindex> <propertyindex>1</propertyindex> </range> </track> i want grep type="ps" , till </range> . one solution open file, read line line , match block. open(fh, "file.txt"); foreach $line (<fh>) { if ($line =~ m/type="cc"(.*?)<\/range>/) { print $1; } } but there more optimal solution without reading file line l
Read more

delphi - How to remove all the grips on a coolbar if I have several coolbands? -

-
Image
if set mycoolbar.fixedorder true,only grip on first band hidden. well,if use delphi 7 create vcl forms application,then put coolbar on , create 3 coolbans hold other controls,only grip , top coolbands can hidden setting mycoolbar.fixedorder:=true. i've uploaded picture make things clear. you got fixedorder property wrong. fixed property not allow bands rearranged if set true . setting property of coolbar true keep user changing bands order @ runtime, user can still move , resize bands. can give advice can do, actual solution problem, well, have wait answer. my advice use 3 coolbars in row , setting "fixedorder" property true , bandborderstyle bsnone . way grip hidden on of them. about property, it's not bug of ide, actual preference of property.
Read more

javascript - Animating array of divs; only the final element is modified -

-
this baffling me @ moment. i'm experienced programmer looking throw bit of surprise fiancee on countdown page our wedding. desired effect simulated confetti falling behind main page. setup follows: i have array of divs in javascript. each div absolutely positioned on page. have interval set updates every 50 or milliseconds. on each tick, run loop on array of divs, , calculations , update positions. problem problem having, however, though can see divs created, stored array, , modified (in case, each has slight randomized rotation) during initialization phase, once tick update position starts, div stored in whatever last slot of array gets position updated. tried hard-coding each div index testing purposes, strictly last element update position. upon printing current "left" , "top" style values each div before , after calculations shows value has been changed, no visible change noticeable. here's basic code: javascript var container;
Read more