PoshCode Logo PowerShell Code Repository

Get-WebFile 3.5 (modification of post by Joel Bennett view diff)
View followups from Joel Bennett | diff | embed code: <script type="text/javascript" src="http://PoshCode.org/embed/189"></script>download | new post

A complete rewrite of my wget script to use HttpWebRequest and Response to figure out the filename, added some unwrapping because a couple sites had quotes around the file names.

  1. ## Get-WebFile.ps1 (aka wget for PowerShell)
  2. ##############################################################################################################
  3. ## Downloads a file or page from the web
  4. ## History:
  5. ## v3.5 - Add -Quiet switch to turn off the progress reports ...
  6. ## v3.4 - Add progress report for files which don't report size
  7. ## v3.3 - Add progress report for files which report their size
  8. ## v3.2 - Use the pure Stream object because StreamWriter is based on TextWriter:
  9. ##        it was messing up binary files, and making mistakes with extended characters in text
  10. ## v3.1 - Unwrap the filename when it has quotes around it
  11. ## v3   - rewritten completely using HttpWebRequest + HttpWebResponse to figure out the file name, if possible
  12. ## v2   - adds a ton of parsing to make the output pretty
  13. ##        added measuring the scripts involved in the command, (uses Tokenizer)
  14. ##############################################################################################################
  15. #function wget {
  16.    param(
  17.       $url = (Read-Host "The URL to download"),
  18.       $fileName,
  19.       [switch]$quiet
  20.    )
  21.    
  22.    $req = [System.Net.HttpWebRequest]::Create($url);
  23.    $res = $req.GetResponse();
  24.  
  25.    if($fileName -and !(Split-Path $fileName)) {
  26.       $fileName = Join-Path (Get-Location -PSProvider "FileSystem") $fileName
  27.    }
  28.    elseif(($fileName -eq $null) -or (Test-Path -PathType "Container" $fileName))
  29.    {
  30. #  if( -and !((Test-Path -PathType "Leaf" $fileName) -or ((Test-Path -PathType "Container" (Split-Path $fileName)) -and -not )))
  31.       [string]$fileName = ([regex]'(?i)filename=(.*)$').Match( $res.Headers["Content-Disposition"] ).Groups[1].Value
  32.       $fileName = $fileName.trim("\/""'")
  33.       if(!$fileName) {
  34.          $fileName = $res.ResponseUri.Segments[-1]
  35.          $fileName = $fileName.trim("\/")
  36.          if(!$fileName) {
  37.             $fileName = Read-Host "Please provide a file name"
  38.          }
  39.          $fileName = $fileName.trim("\/")
  40.          if(!([IO.FileInfo]$fileName).Extension) {
  41.             $fileName = $fileName + "." + $res.ContentType.Split(";")[0].Split("/")[1]
  42.          }
  43.       }
  44.       $fileName = Join-Path (Get-Location -PSProvider "FileSystem") $fileName
  45.    }
  46.  
  47.    if($res.StatusCode -eq 200) {
  48.       [int]$goal = $res.ContentLength
  49.       $reader = $res.GetResponseStream()
  50.       $writer = new-object System.IO.FileStream $fileName, "Create"
  51.       [byte[]]$buffer = new-object byte[] 4096
  52.       [int]$total = [int]$count = 0
  53.       do
  54.       {
  55.          $count = $reader.Read($buffer, 0, $buffer.Length);
  56.          $writer.Write($buffer, 0, $count);
  57.          if(!$quiet) {
  58.             $total += $count
  59.             if($goal -gt 0) {
  60.                Write-Progress "Downloading $url" "Saving $total of $goal" -id 0 -percentComplete (($total/$goal)*100)
  61.             } else {
  62.                Write-Progress "Downloading $url" "Saving $total bytes..." -id 0
  63.             }
  64.          }
  65.       } while ($count -gt 0)
  66.      
  67.       $writer.Flush()
  68.       $reader.Close()
  69.       $writer.Close()
  70.    }
  71.    $res.Close();
  72.    ls $fileName
  73. #}

Submit a correction or amendment below (click here to make a fresh posting)
After submitting an amendment, you'll be able to view the differences between the old and new posts easily.

Syntax highlighting:


Remember me