python - Is there a better way to do csv/namedtuple with urlopen? -

-

using namedtuple documentation example template in python 3.3, have following code download csv , turn series of namedtuple subclass instances:

from collections import namedtuple csv import reader urllib.request import urlopen      securitytype = namedtuple('securitytype', 'sector, name')  url = 'http://bsym.bloomberg.com/sym/pages/security_type.csv' sec in map(securitytype._make, reader(urlopen(url))):     print(sec)

this raises following exception:

traceback (most recent call last):   file "scrap.py", line 9, in <module>     sec in map(securitytype._make, reader(urlopen(url))): _csv.error: iterator should return strings, not bytes (did open file in text mode?)

i know issue urlopen returning bytes , not strings , need decode output @ point. here's how i'm doing now, using stringio:

from collections import namedtuple csv import reader urllib.request import urlopen import io  securitytype = namedtuple('securitytype', 'sector, name')  url = 'http://bsym.bloomberg.com/sym/pages/security_type.csv' reader_input = io.stringio(urlopen(url).read().decode('utf-8'))  sec in map(securitytype._make, reader(reader_input)):     print(sec)

this smells funny because i'm iterating on bytes buffer, decoding, rebuffering, iterating on new string buffer. there more pythonic way without 2 iterations?

use io.textiowrapper() decode urllib response:

reader_input = io.textiowrapper(urlopen(url), encoding='utf8', newline='')

now csv.reader passed exact same interface when opening regular file on filesystem in text mode.

with change example url works me on python 3.3.1:

>>> sec in map(securitytype._make, reader(reader_input)): ...     print(sec) ...  securitytype(sector='market sector', name='security type') securitytype(sector='comdty', name='calendar spread option') securitytype(sector='comdty', name='financial commodity future.') securitytype(sector='comdty', name='financial commodity generic.') securitytype(sector='comdty', name='financial commodity option.') ... securitytype(sector='muni', name='zero coupon, oid') securitytype(sector='pfd', name='private') securitytype(sector='pfd', name='public') securitytype(sector='', name='') securitytype(sector='', name='') securitytype(sector='', name='') securitytype(sector='', name='') securitytype(sector='', name='') securitytype(sector='', name='') securitytype(sector='', name='') securitytype(sector='', name='') securitytype(sector='', name='')

the last lines appear yield empty tuples; original indeed has lines nothing more comma on them.


Comments

Post a Comment

Popular posts from this blog

Perl - how to grep a block of text from a file -

-
it can in xml or text format. how in general grep block of text in perl? <track type="ws"> <range> <rangestart>0</rangestart> <rangeend>146.912</rangeend> <locationindex>0</locationindex> <propertyindex>0</propertyindex> </range> </track> <track type="ps" id="1"> <range> <rangestart>0</rangestart> <rangeend>146.912</rangeend> <locationindex>1</locationindex> <propertyindex>1</propertyindex> </range> </track> i want grep type="ps" , till </range> . one solution open file, read line line , match block. open(fh, "file.txt"); foreach $line (<fh>) { if ($line =~ m/type="cc"(.*?)<\/range>/) { print $1; } } but there more optimal solution without reading file line l
Read more

delphi - How to remove all the grips on a coolbar if I have several coolbands? -

-
Image
if set mycoolbar.fixedorder true,only grip on first band hidden. well,if use delphi 7 create vcl forms application,then put coolbar on , create 3 coolbans hold other controls,only grip , top coolbands can hidden setting mycoolbar.fixedorder:=true. i've uploaded picture make things clear. you got fixedorder property wrong. fixed property not allow bands rearranged if set true . setting property of coolbar true keep user changing bands order @ runtime, user can still move , resize bands. can give advice can do, actual solution problem, well, have wait answer. my advice use 3 coolbars in row , setting "fixedorder" property true , bandborderstyle bsnone . way grip hidden on of them. about property, it's not bug of ide, actual preference of property.
Read more

javascript - Animating array of divs; only the final element is modified -

-
this baffling me @ moment. i'm experienced programmer looking throw bit of surprise fiancee on countdown page our wedding. desired effect simulated confetti falling behind main page. setup follows: i have array of divs in javascript. each div absolutely positioned on page. have interval set updates every 50 or milliseconds. on each tick, run loop on array of divs, , calculations , update positions. problem problem having, however, though can see divs created, stored array, , modified (in case, each has slight randomized rotation) during initialization phase, once tick update position starts, div stored in whatever last slot of array gets position updated. tried hard-coding each div index testing purposes, strictly last element update position. upon printing current "left" , "top" style values each div before , after calculations shows value has been changed, no visible change noticeable. here's basic code: javascript var container;
Read more