Failure in get_bucket

[nhammond]

Command failed once after hanging for a minute or two. It succeeded when I retried.

loom import file gs://storage-public-data/WorkflowInputFiles/hg19/references/ucsc.hg19.fasta

Traceback (most recent call last):
  File "/usr/local/bin/loom", line 9, in <module>
    load_entry_point('loomengine==0.3.3', 'console_scripts', 'loom')()
  File "/home/s0198839/loom/loomengine/client/main.py", line 89, in main
    return Main().run()
  File "/home/s0198839/loom/loomengine/client/main.py", line 85, in run
    return self.args.SubcommandClass(self.args).run()
  File "/home/s0198839/loom/loomengine/client/importer.py", line 160, in run
    self.args.SubSubcommandClass(self.args).run()
  File "/home/s0198839/loom/loomengine/client/importer.py", line 47, in run
    force_duplicates=self.args.force_duplicates,
  File "/home/s0198839/loom/loomengine/utils/filemanager.py", line 454, in import_from_patterns
    pattern, note, force_duplicates=force_duplicates))
  File "/home/s0198839/loom/loomengine/utils/filemanager.py", line 459, in import_from_pattern
    for source in SourceSet(pattern, self.settings):
  File "/home/s0198839/loom/loomengine/utils/filemanager.py", line 41, in SourceSet
    return GoogleStorageSourceSet(pattern, settings)
  File "/home/s0198839/loom/loomengine/utils/filemanager.py", line 95, in __init__
    self.sources = [GoogleStorageSource(pattern, settings)]
  File "/home/s0198839/loom/loomengine/utils/filemanager.py", line 208, in __init__
    self.bucket = self.client.get_bucket(self.bucket_id)
  File "/usr/local/lib/python2.7/dist-packages/gcloud/storage/client.py", line 167, in get_bucket
    bucket.reload(client=self)
  File "/usr/local/lib/python2.7/dist-packages/gcloud/storage/_helpers.py", line 77, in reload
    _target_object=self)
  File "/usr/local/lib/python2.7/dist-packages/gcloud/connection.py", line 343, in api_request
    target_object=_target_object)
  File "/usr/local/lib/python2.7/dist-packages/gcloud/connection.py", line 241, in _make_request
    return self._do_request(method, url, headers, data, target_object)
  File "/usr/local/lib/python2.7/dist-packages/gcloud/connection.py", line 270, in _do_request
    body=data)
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/transport.py", line 169, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1659, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1399, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1355, in _conn_request
    response = conn.getresponse()
  File "/usr/lib/python2.7/httplib.py", line 1136, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 409, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/lib/python2.7/ssl.py", line 756, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 643, in read
    v = self._sslobj.read(len)
socket.error: [Errno 104] Connection reset by peer

[nhammond]

All calls to Google service APIs should retry.
#369

[nhammond]

Added retries with exponential backoff for get_bucket and get_blob. 1a2ada9