amazon s3 - Python Write Temp File to S3 -


i trying write dataframe temp file , upload temp file s3 bucket. when run code there isn't action occurs. appreciated. following code:

import csv import pandas pd import boto3 import tempfile import os    temp = tempfile.temporaryfile() largedf.to_csv(temp, sep = '|') s3.put_object(temp, bucket = '[bucket name]', key = 'test.txt') temp.close() 

the file-handle pass s3.put_object @ final position, when .read it, return empty string.

>>> df = pd.dataframe(np.random.randint(10,50, (5,5))) >>> temp = tempfile.temporaryfile(mode='w+') >>> df.to_csv(temp) >>> temp.read() '' 

a quick fix .seek beginning...

>>> temp.seek(0) 0 >>> print(temp.read()) ,0,1,2,3,4 0,11,42,40,45,11 1,36,18,45,24,25 2,28,20,12,33,44 3,45,39,14,16,20 4,40,16,22,30,37 

note, writing disk unnecessary, really, keep in memory using buffer, like:

from io import stringio # on python 2, use cstringio import stringio buffer = stringio() pd.to_csv(buffer) buffer.seek(0) s3.put_object(buffer, bucket = '[bucket name]', key = 'test.txt') 

Comments

Popular posts from this blog

ZeroMQ on Windows, with Qt Creator -

unity3d - Unity SceneManager.LoadScene quits application -

python - Error while using APScheduler: 'NoneType' object has no attribute 'now' -