python - Google Earth Engine : Exporting MODIS images from GEE to AWS S3 bucket -

i working on machine learning project uses modis dataset. pc doesn't meet computational requirements of project, had taken aws server. problem earth engine exporting images google drive or google cloud storage want them exported s3 bucket.

i have come across answers suggesting download data local storage , upload them s3 bucket. given huge datasets , poor data speed, take me ages so. hence want export them s3 bucket directly using earth engine.

i have gone through documentation exporting happens (ee.batch.export.image). thinking of writing function exports geotiff images aws s3 bucket instead of google drive or cloud storage.

p.s.

• i have verified amazon modis public datasets , datasets want (mod09a1 , few others) aren't offered amazon.
• i have windows 10 installed on pc.

modis imagery on aws s3 (https://aws.amazon.com/public-datasets/modis/)

however, interesting question other data set , here few things consider

1) google earth engine can write onto google cloud storage buckets (gcs) free upto 5 gb or google drive has limit of 15 gb including gmail. able download these images local drive before pushing aws s3 need make sure have enough space available on either gcs or drive.

2) google earth engine not export metadata , split large geotiff if exceeds file size limits(certain things keep in mind) incase want mosaic split images single image before uploading aws. can export properties metadata csv or kml file.

3) once know have enough buffer space in gcs or google drive ideal method be

• push ee drive/gcs
• pull drive/gcs local , push aws (if wanted using google's network speed instead of client resource can spin small micro instance under always free model google)

• another way without using client resource using web integration service, example, zapier can link drive aws new files come in copied aws , can use new file coming in trigger). [i have not tried know can done using zapier or ifttt.

• periodically check cloud storage , delete files copied onto gcs can have check if files or folders synced , delete files on gcs/drive free space again , repeat process.

i sure others might have other great suggestions way of doing it. hope helps

sam