Research
Set timeout for urllib urlretrieve download files
Shakeratto
2018. 8. 18. 08:02
1. Basic code of urlretrieve to download a file
urllib.request.urlretrieve(source_url, destination_path)
2. Advanced code of urlretrive to display the download percentage.
def download_file(source_url, destination_path):
"""Downloads `source_url` onto `destionation_path`."""
def _progress(count, block_size, total_size):
sys.stderr.write('\r>> Downloading %s %.1f%%' % (
source_url, float(count * block_size) / float(total_size) * 100.0))
sys.stderr.flush()
urllib.request.urlretrieve(source_url, destination_path, _progress)
3. Set timeout for urlretrive when download is stopping
(Below code can be placed on the top of the code)
(30 seconds)
import socket
socket.setdefaulttimeout(30)
4.Set retries count
def download_file(source_url, destination_path, retries=3):
def _progress(count, block_size, total_size):
sys.stderr.write('\r>> Downloading %s %.1f%%' % (
source_url, float(count * block_size) / float(total_size) * 100.0))
sys.stderr.flush()
while(retries > 0):
try:
urllib.request.urlretrieve(source_url, destination_path, _progress)
break
except:
print("retry")
retries = retries - 1
continue