python - use a list outside my loop to concatenate a url string with some list values -
im getting 3 messages sqs queue: mapper.py, reducer.py , test.txt ...and im storing each message inside list.
and want use content of each message in mapper, reducer , input variables(last 3 code lines).
but outside while loop line have print "connecting emr"
im not seeing how can use list of filenames purpose want.
do see solution this?
in print item
inside loop shows first element of filenames_list.
conn = sqsconnection() myqueue = conn.get_queue('myqueue') while myqueue.count() != 0: filenames_list = list() message = myqueue.read() myqueue.delete_message(message) filename = message.get_body() filenames_list.append(filename) item in files_list: print item print "connecting emr" conn = boto.emr.connect_to_region('us-east-1') print "creating streaming step" step = streamingstep(name='example1', # after myfolder/ want put here content of list relative test.txt mapper="s3n://myfolder/hereiwanttoputthelistcontentrelatedtomapper.py", # after myfolder/ want put here content of list relative reducer.py reducer="s3n://myfolder/hereiwanttoputthelistcontentrelatedtoreducer.py", # after myfolder/ want put here content of list relative mapper.py input='s3n://myfolder/hereiwanttoputthelistcontentrelatedtotest.txt', output='s3n://myfolder/output/wordcount_output')
you're initializing files_list empty list @ start of each iteration of while_loop, perhaps want initialize outside list , append on each iteration of while loop?
Comments
Post a Comment