Google Chrome 17 BETA – Download Protection Feature

Recently Google Chrome 17 BETA announced two new features. One of them is the download security. So i started to look into it. You can read their blog in this link. "Speed and Security" (http://chrome.blogspot.com/2012/01/speed-and-security.html)

As they mentioned in the blog, it uses Google SafeBrowsing Service for the malicious URL/File detection. In simple what it does is, it does a partial match on the URL/File locally and then query Google for the Full Hash. It uses HTTP protocol to exchange the hash details.

This detection technique goes like this:
    1. User clicks a download link and Chrome detects this as download link. It keep track of the URL chain to reach the final download URL.
    2. Before starting the file download, it checks the URL Chain and final download URL using SafeBrowsing Service.
        2.1 If the final download link or one of the link in the chain is a malicious URL then alert the user.
        2.2 Otherwise start the download
    3. Once the download is finished, find SHA-256 hash of the file and check it using SafeBrowsing Service.
        3.1 If the downloaded file is a malicious file then alert the user otherwise finish the file download process.

How the detection works ?
    We give an URL or file hash to the SafeBrowsing Service. It initially does Partial hash test first and if it succeed then it will do a full hash test by querying Google. So it is a cloud based detection mechanism. Google keeps a limited information in the PC and query Google for the final result. In the worst case, assume file/url has passed the partial match and we couldn’t able to query Google because of the network issue then Chrome considers this file as SAFE. In any worst case, Google Chrome assumes that file is SAFE and it makes sense.

We need to divide our study into two different section:
    1. Download URL Check
    2. Download Hash Check

In this blog post we will just concentrate on the “Download URL Check”

C++ Classes we get to see are:
    DownloadManagerImpl                        -> Chrome Download Manager Implementation
    ChromeDownloadManagerDelegate  -> Chrome Download Manager gives full responsibility of file download to this class.
    DownloadProtectionService                -> Implements the SafeBrowsing Clients
    DownloadUrlSBClient                           -> Implements the SafeBrowsing URL Check Client
    SafeBrowsingService                            -> Handles SafeBrowsing feature.

Control Flow:
DownloadManagerImpl receives an notification from browser for any new download request. It pass the control to the ChromeDownloadManagerDelegate class which handles the file download. ChromeDownloadManagerDelegate checks with DownloadProtectionService whether the URL/URL Chain is SAFE Or not. DownloadProtectionService creates an client of type DownloadUrlSBClient and gives the control to it. DownloadUrlSBClient uses SafeBrowsingService to do the final check. Once SafeBrowsingService has a result, it calls the DownloadUrlSBClient with the result and it in turn gives the result back to ChromeDownloadManagerDelegate, which marks the download URL as SAFE or not. In turn browser process alerts the user about the malicious URL.

Ok, in more depth….

DownloadManagerImpl::StartDownload() receives control when the user started a download event. DownloadManagerImpl::StartDownload() delegates this download event to ChromeDownloadManagerDelegate::ShouldStartDownload() which creates an object of type SafeBrowsingService if it is not created already. This SafeBrowsingService object creates an object of type DownloadProtectionService internally. So there is one-to-one relation between SafeBrowsingService and DownloadProtectionService. ChromeDownloadManagerDelegate::ShouldStartDownload() calls DownloadProtectionService::CheckDownloadUrl() with download information and a callback to return when it have the result. DownloadProtectionService::CheckDownloadUrl() creates an SafeBrowsing “Client” of type DownloadUrlSBClient and gives the full responsibility to it. DownloadUrlSBClient has all information about download, callback to return and which SafeBrowsingService to use. DownloadUrlSBClient starts its work from DownloadUrlSBClient::StartCheck(). DownloadUrlSBClient::StartCheck() ask SafeBrowsing Service about the intent of URL. It calls SafeBrowsingService::CheckDownloadUrl() with the download information and Client to report back. Once SafeBrowsingService finish it’s work then it calls SafeBrowsingService::SafeBrowsingCheckDone() with the result. SafeBrowsingService in turn calls the Client’s DownloadUrlSBClient::OnDownloadUrlCheckResult() which in turn calls DownloadUrlSBClient::CheckDone(). Now client finished its work and it has the result also. So it will callChromeDownloadManagerDelegate::CheckDownloadUrlDone() which marks the URL accordingly.

DownloadManagerImpl -> ChromeDownloadManagerDelegate -> DownloadProtectionService -> DownloadUrlSBClient -> SafeBrowsingService -> [does all the check] -> SafeBrowsingService -> DownloadUrlSBClient -> ChromeDownloadManagerDelegate

Source Code:
             // We have received a message from DownloadFileManager about a new download.
             void DownloadManagerImpl::StartDownload(int32 download_id)
            {
                DCHECK(BrowserThread::CurrentlyOn(BrowserThread::UI));

  if (delegate_->ShouldStartDownload(download_id))
    RestartDownload(download_id);
}

bool ChromeDownloadManagerDelegate::ShouldStartDownload(int32 download_id)
{
  // We create a download item and store it in our download map, and inform the
  // history system of a new download. Since this method can be called while the
  // history service thread is still reading the persistent state, we do not
  // insert the new DownloadItem into ‘history_downloads_’ or inform our
  // observers at this point. OnCreateDownloadEntryComplete() handles that
  // finalization of the the download creation as a callback from the history
  // thread.
  DownloadItem* download = download_manager_->GetActiveDownloadItem(download_id);
  if (!download)
    return false;

#if defined(ENABLE_SAFE_BROWSING)
  DownloadProtectionService* service = GetDownloadProtectionService();
  if (service) {
    VLOG(2) << __FUNCTION__ << "() Start SB URL check for download = " << download->DebugString(false);
    service->CheckDownloadUrl(DownloadProtectionService::DownloadInfo::FromDownloadItem(*download),base::Bind(&ChromeDownloadManagerDelegate::CheckDownloadUrlDone,this,download->GetId()));
    return false;
  }
#endif
  CheckDownloadUrlDone(download_id, DownloadProtectionService::SAFE);
  return false;
}

void DownloadProtectionService::CheckDownloadUrl(const DownloadProtectionService::DownloadInfo& info,const CheckDownloadCallback& callback)
{
  DCHECK(!info.download_url_chain.empty());
  scoped_refptr<DownloadUrlSBClient> client(new DownloadUrlSBClient(info, callback, sb_service_));
 
  // The client will release itself once it is done.
  BrowserThread::PostTask(BrowserThread::IO,FROM_HERE,base::Bind(&DownloadUrlSBClient::StartCheck, client));                    // POSTTASK
}

virtual void DownloadUrlSBClient::StartCheck() OVERRIDE
{
    DCHECK(BrowserThread::CurrentlyOn(BrowserThread::IO));
    if (!sb_service_ || sb_service_->CheckDownloadUrl(info_.download_url_chain, this)) {
      CheckDone(SafeBrowsingService::SAFE);
    } else {
      AddRef();  // SafeBrowsingService takes a pointer not a scoped_refptr.
    }
}

bool SafeBrowsingService::CheckDownloadUrl(const std::vector<GURL>& url_chain,Client* client)
{
  DCHECK(BrowserThread::CurrentlyOn(BrowserThread::IO));
  if (!enabled_ || !enable_download_protection_)
    return true;

  // We need to check the database for url prefix, and later may fetch the url
  // from the safebrowsing backends. These need to be asynchronous.
  SafeBrowsingCheck* check = new SafeBrowsingCheck();
  check->urls = url_chain;
  StartDownloadCheck(check,client,base::Bind(&SafeBrowsingService::CheckDownloadUrlOnSBThread, this, check), download_urlcheck_timeout_ms_);
  return false;
}

void SafeBrowsingService::StartDownloadCheck(SafeBrowsingCheck* check,Client* client,const base::Closure& task,int64 timeout_ms)
{
  DCHECK(BrowserThread::CurrentlyOn(BrowserThread::IO));
  check->client = client;
  check->result = SAFE;
  check->is_download = true;
  check->timeout_factory_.reset(new base::WeakPtrFactory<SafeBrowsingService>(this));
  checks_.insert(check);

  safe_browsing_thread_->message_loop()->PostTask(FROM_HERE, task);

  MessageLoop::current()->PostDelayedTask(FROM_HERE,base::Bind(&SafeBrowsingService::TimeoutCallback,check->timeout_factory_->GetWeakPtr(), check),timeout_ms);
}

void SafeBrowsingService::CheckDownloadUrlOnSBThread(SafeBrowsingCheck* check)
{
  DCHECK_EQ(MessageLoop::current(), safe_browsing_thread_->message_loop());
  DCHECK(enable_download_protection_);

  std::vector<SBPrefix> prefix_hits;

  if (!database_->ContainsDownloadUrl(check->urls, &prefix_hits)) {
    // Good, we don’t have hash for this url prefix.
    check->result = SAFE;
    BrowserThread::PostTask(BrowserThread::IO, FROM_HERE,base::Bind(&SafeBrowsingService::CheckDownloadUrlDone, this, check));
    return;
  }

  check->need_get_hash = true;
  check->prefix_hits.clear();
  check->prefix_hits = prefix_hits;
  BrowserThread::PostTask(BrowserThread::IO, FROM_HERE,base::Bind(&SafeBrowsingService::OnCheckDone, this, check));
}

void SafeBrowsingService::OnCheckDone(SafeBrowsingCheck* check)
{
  DCHECK(BrowserThread::CurrentlyOn(BrowserThread::IO));

  if (!enabled_)
    return;

  // If the service has been shut down, |check| should have been deleted.
  DCHECK(checks_.find(check) != checks_.end());

  if (check->client && check->need_get_hash) {
    // We have a partial match so we need to query Google for the full hash.
    // Clean up will happen in HandleGetHashResults.

    // See if we have a GetHash request already in progress for this particular
    // prefix. If so, we just append ourselves to the list of interested parties
    // when the results arrive. We only do this for checks involving one prefix,
    // since that is the common case (multiple prefixes will issue the request
    // as normal).
    if (check->prefix_hits.size() == 1) {
      SBPrefix prefix = check->prefix_hits[0];
      GetHashRequests::iterator it = gethash_requests_.find(prefix);
      if (it != gethash_requests_.end()) {
        // There’s already a request in progress.
        it->second.push_back(check);
        return;
      }

      // No request in progress, so we’re the first for this prefix.
      GetHashRequestors requestors;
      requestors.push_back(check);
      gethash_requests_[prefix] = requestors;
    }

    // Reset the start time so that we can measure the network time without the
    // database time.
    check->start = base::TimeTicks::Now();
    protocol_manager_->GetFullHash(check, check->prefix_hits);
  } else {
    // We may have cached results for previous GetHash queries.  Since
    // this data comes from cache, don’t histogram hits.
    HandleOneCheck(check, check->full_hits);
  }
}

void SafeBrowsingProtocolManager::GetFullHash(SafeBrowsingService::SafeBrowsingCheck* check,const std::vector<SBPrefix>& prefixes)
{
  // If we are in GetHash backoff, we need to check if we’re past the next
  // allowed time. If we are, we can proceed with the request. If not, we are
  // required to return empty results (i.e. treat the page as safe).
  if (gethash_error_count_ && Time::Now() <= next_gethash_time_) {
    std::vector<SBFullHashResult> full_hashes;
    sb_service_->HandleGetHashResults(check, full_hashes, false);
    return;
  }
  bool use_mac = !client_key_.empty();
  GURL gethash_url = GetHashUrl(use_mac);
  content::URLFetcher* fetcher = content::URLFetcher::Create(gethash_url, content::URLFetcher::POST, this);
  hash_requests_[fetcher] = check;

  std::string get_hash;
  SafeBrowsingProtocolParser parser;
  parser.FormatGetHash(prefixes, &get_hash);

  fetcher->SetLoadFlags(net::LOAD_DISABLE_CACHE);
  fetcher->SetRequestContext(request_context_getter_);
  fetcher->SetUploadData("text/plain", get_hash);
  fetcher->Start();
}

void SafeBrowsingService::HandleGetHashResults(SafeBrowsingCheck* check,const std::vector<SBFullHashResult>& full_hashes,bool can_cache)
{
  DCHECK(BrowserThread::CurrentlyOn(BrowserThread::IO));

  if (!enabled_)
    return;

  // If the service has been shut down, |check| should have been deleted.
  DCHECK(checks_.find(check) != checks_.end());

  // |start| is set before calling |GetFullHash()|, which should be
  // the only path which gets to here.
  DCHECK(!check->start.is_null());
  UMA_HISTOGRAM_LONG_TIMES("SB2.Network",base::TimeTicks::Now() – check->start);

  std::vector<SBPrefix> prefixes = check->prefix_hits;
  OnHandleGetHashResults(check, full_hashes);  // ‘check’ is deleted here.

  if (can_cache && MakeDatabaseAvailable()) {
    // Cache the GetHash results in memory:
    database_->CacheHashResults(prefixes, full_hashes);
  }
}

void SafeBrowsingService::OnHandleGetHashResults(SafeBrowsingCheck* check,const std::vector<SBFullHashResult>& full_hashes)
{
  DCHECK(BrowserThread::CurrentlyOn(BrowserThread::IO));
  bool is_download = check->is_download;
  SBPrefix prefix = check->prefix_hits[0];
  GetHashRequests::iterator it = gethash_requests_.find(prefix);
  if (check->prefix_hits.size() > 1 || it == gethash_requests_.end()) {
    const bool hit = HandleOneCheck(check, full_hashes);
    RecordGetHashCheckStatus(hit, is_download, full_hashes);
    return;
  }

  // Call back all interested parties, noting if any has a hit.
  GetHashRequestors& requestors = it->second;
  bool hit = false;
  for (GetHashRequestors::iterator r = requestors.begin(); r != requestors.end(); ++r) {
    if (HandleOneCheck(*r, full_hashes))
        hit = true;
  }
  RecordGetHashCheckStatus(hit, is_download, full_hashes);

  gethash_requests_.erase(it);
}

bool SafeBrowsingService::HandleOneCheck(SafeBrowsingCheck* check,const std::vector<SBFullHashResult>& full_hashes)
{
  DCHECK(BrowserThread::CurrentlyOn(BrowserThread::IO));
  DCHECK(check);

  // Always calculate the index, for recording hits.
  int index = -1;
  if (!check->urls.empty()) {
    for (size_t i = 0; i < check->urls.size(); ++i) {
      index = safe_browsing_util::GetUrlHashIndex(check->urls[i], full_hashes);
      if (index != -1)
        break;
    }
  } else {
    index = safe_browsing_util::GetHashIndex(*(check->full_hash), full_hashes);
  }

  // |client| is NULL if the request was cancelled.
  if (check->client) {
    check->result = SAFE;
    if (index != -1)
      check->result = GetResultFromListname(full_hashes[index].list_name);
  }
  SafeBrowsingCheckDone(check);
  return (index != -1);
}

SafeBrowsingService::UrlCheckResult SafeBrowsingService::GetResultFromListname(const std::string& list_name)
{
  if (safe_browsing_util::IsPhishingList(list_name)) {
    return URL_PHISHING;
  }

  if (safe_browsing_util::IsMalwareList(list_name)) {
    return URL_MALWARE;
  }

  if (safe_browsing_util::IsBadbinurlList(list_name)) {
    return BINARY_MALWARE_URL;
  }

  if (safe_browsing_util::IsBadbinhashList(list_name)) {
    return BINARY_MALWARE_HASH;
  }

  DVLOG(1) << "Unknown safe browsing list " << list_name;
  return SAFE;
}

void SafeBrowsingService::CheckDownloadUrlDone(SafeBrowsingCheck* check)
{
  DCHECK(enable_download_protection_);
  SafeBrowsingCheckDone(check);
}

void SafeBrowsingService::SafeBrowsingCheckDone(SafeBrowsingCheck* check)
{
  DCHECK(BrowserThread::CurrentlyOn(BrowserThread::IO));
  DCHECK(check);

  if (!enabled_)
    return;

  VLOG(1) << "SafeBrowsingCheckDone: " << check->result;
  DCHECK(checks_.find(check) != checks_.end());
  if (check->client)
    check->client->OnSafeBrowsingResult(*check);
  checks_.erase(check);
  delete check;
}

void SafeBrowsingService::Client::OnSafeBrowsingResult(const SafeBrowsingCheck& check)
{
  if (!check.urls.empty()) {
    DCHECK(!check.full_hash.get());
    if (!check.is_download) {
        DCHECK_EQ(1U, check.urls.size());
        OnBrowseUrlCheckResult(check.urls[0], check.result);
    } else {
        OnDownloadUrlCheckResult(check.urls, check.result);
    }
  } else if (check.full_hash.get()) {
        OnDownloadHashCheckResult(safe_browsing_util::SBFullHashToString(*check.full_hash),check.result);
  } else {
    NOTREACHED();
  }
}

virtual void DownloadUrlSBClient::OnDownloadUrlCheckResult(const std::vector<GURL>& url_chain,SafeBrowsingService::UrlCheckResult sb_result) OVERRIDE
{
    CheckDone(sb_result);
    UMA_HISTOGRAM_TIMES("SB2.DownloadUrlCheckDuration", base::TimeTicks::Now() – start_time_);
    Release();
}

void DownloadUrlSBClient::CheckDone(SafeBrowsingService::UrlCheckResult sb_result)
{
    DownloadProtectionService::DownloadCheckResult result = IsDangerous(sb_result) ? DownloadProtectionService::DANGEROUS : DownloadProtectionService::SAFE;
    BrowserThread::PostTask(BrowserThread::UI,FROM_HERE,base::Bind(callback_, result));                                        // POSTTASK
    UpdateDownloadCheckStats(total_type_);
    if (sb_result != SafeBrowsingService::SAFE) {
      UpdateDownloadCheckStats(dangerous_type_);
      BrowserThread::PostTask(BrowserThread::UI,FROM_HERE,base::Bind(&DownloadSBClient::ReportMalware,this, sb_result));       // POSTTASK
    }
}

void ChromeDownloadManagerDelegate::CheckDownloadUrlDone(int32 download_id,DownloadProtectionService::DownloadCheckResult result)
{
  DCHECK(BrowserThread::CurrentlyOn(BrowserThread::UI));
  DownloadItem* download = download_manager_->GetActiveDownloadItem(download_id);
  if (!download)
    return;

  VLOG(2) << __FUNCTION__ << "() download = " << download->DebugString(false) << " verdict = " << result;
  if (result == DownloadProtectionService::DANGEROUS)
    download->MarkUrlDangerous();

  download_history_->CheckVisitedReferrerBefore(download_id, download->GetReferrerUrl(),base::Bind(&ChromeDownloadManagerDelegate::CheckVisitedReferrerBeforeDone,base::Unretained(this)));
}

Conclusion
    1. Google Collects lot of information about the type of files we download and how many of them are with malicous intent.
    2. Google Uses MAC as a SafeBrowsing key when it query Google. (If no SafeBrowsing key is provided.)

Advertisements
This entry was posted in browser, Chrome, Cr-48, Internals and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s