將大文件從驅動器下載到 colab (download large file from drive to colab)

問題描述

將大文件從驅動器下載到 colab (download large file from drive to colab)

我有一個指向公共 Google 雲端硬盤託管文件的鏈接：

https://drive.google.com/uc?id=19VsarMcYRNPLTDr6b6ABJyY8JUeBueL8&export=download

以下是適用於不同文件和鏈接的 .sh 腳本：

#!/usr/bin/env bash
function gdrive_download () { # credit to https://github.com/ethanjperez/convince
  CONFIRM=$(wget ‑‑quiet ‑‑save‑cookies /tmp/cookies.txt ‑‑keep‑session‑cookies ‑‑no‑check‑certificate "https://docs.google.com/uc?export=download&id=$1" ‑O‑ | sed ‑rn 's/.*confirm=([0‑9A‑Za‑z_]+).*/\1\n/p')
  wget ‑‑load‑cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$CONFIRM&id=$1" ‑O $2
  rm ‑rf /tmp/cookies.txt
}

mkdir ‑p Models/real‑fixed‑cam Models/real‑hand‑held
gdrive_download 1yiNsSkPYoBZ55fSQ1iwb1io9QL_PcR2i Models/real‑fixed‑cam/netG_epoch_12.pth
gdrive_download 13HckO9fPAKYocdB_CAC5n8uyM3xQ2MpG Models/real‑hand‑held/netG_epoch_12.pth

上面的腳本在 Colab 中被調用：

!wget https://gist.githubusercontent.com/andreyryabtsev/458f7450c630952d1e75e195f94845a0/raw/0b4336ac2a2140ac2313f9966316467e8cd3002a/download.sh
!chmod +x download.sh
!./download.sh

我已經對其進行瞭如下調整以滿足我的需要：

#!/usr/bin/env bash
function gdrive_download () { # credit to https://github.com/ethanjperez/convince
  CONFIRM=$(wget ‑‑quiet ‑‑save‑cookies /tmp/cookies.txt ‑‑keep‑session‑cookies ‑‑no‑check‑certificate "https://docs.google.com/uc?export=download&id=$1" ‑O‑ | sed ‑rn 's/.*confirm=([0‑9A‑Za‑z_]+).*/\1\n/p')
  wget ‑‑load‑cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$CONFIRM&id=$1" ‑O $2
  rm ‑rf /tmp/cookies.txt
}

mkdir ‑p pix2pix/checkpoint
gdrive_download 19VsarMcYRNPLTDr6b6ABJyY8JUeBueL8 pix2pix/checkpoint/weights.zip

上面的代碼被調用colab with:

!wget https://gist.githubusercontent.com/Daryl149/070397c9cb3539f5cd01173f6c44200d/raw/207a76e94e70e6c9334f48c25b4998f4fd1b95e3/download.sh
!chmod +x download.sh
!./download.sh

文件夾已正確創建。但它並沒有將 500mb+ 的 zip 文件下載到 checkpoints 文件夾，而是從下載確認頁面下載了 html。

參考解法

方法 1:

Try this

!gdown 19VsarMcYRNPLTDr6b6ABJyY8JUeBueL8

Then, you can create a new directory with !mkdir or move the weights.zip there.

方法 2:

Based on @korakot's answer, the full working code to achieve the result in Colab is:

!gdown https://drive.google.com/uc?id=19VsarMcYRNPLTDr6b6ABJyY8JUeBueL8
!mkdir /content/Person_remover/pix2pix/checkpoint
import shutil
shutil.move("/content/Person_remover/weights.zip", "/content/Person_remover/pix2pix/checkpoint")

(by DaReal、korakot、DaReal)

參考文件

1. download large file from drive to colab (CC BY‑SA 2.5/3.0/4.0)