Table of Contents

Linux Commands#Network

wget - How to use wget on Linux



man wget

# man wget
       -S
       --server-response
           Print the headers sent by HTTP servers and responses sent by FTP servers.

       --spider
           When invoked with this option, Wget will behave as a Web spider, which means that it will
           not download the pages, just check that they are there. 
           For example, you can use Wget to check your bookmarks:

       -O file
       --output-document=file


       --no-check-certificate
           Don't check the server certificate against the available certificate authorities.


Format

NTML

wget --http-user='DOMAINNAME\USERNAME' --http-password=PASS http://www.example.com/
http-user=DOMAINNAME\USERNAME
http-password=PASS
wget  http://www.example.com/


Examples

How to use proxy server with wget

wget -e http_proxy=xx.xx.xx.xx:8080 http://example.com/ 
wget -e https_proxy=xx.xx.xx.xx:8080 https://example.com/ 


Only Header Information

#wget -S --spider http://www.uniqlo.com/
Spider mode enabled. Check if remote file exists.
--2015-07-29 00:34:06--  http://www.uniqlo.com/
Resolving www.uniqlo.com (www.uniqlo.com)... 23.34.107.227
Connecting to www.uniqlo.com (www.uniqlo.com)|23.34.107.227|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 302 Moved Temporarily
  Server: AkamaiGHost
  Content-Length: 0
  Location: http://www.uniqlo.com/jp/
  Date: Tue, 28 Jul 2015 15:34:07 GMT
  Connection: keep-alive
Location: http://www.uniqlo.com/jp/ [following]
Spider mode enabled. Check if remote file exists.
--2015-07-29 00:34:06--  http://www.uniqlo.com/jp/
Connecting to www.uniqlo.com (www.uniqlo.com)|23.34.107.227|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Server: Apache/2.0.52 (Unix)
  X-Powered-By: PHP/4.3.9
  TestHeader: AWS fr-www3
  Content-Type: text/html
  Content-Length: 196817
  Date: Tue, 28 Jul 2015 15:34:07 GMT
  Connection: keep-alive
  Vary: User-Agent,Accept-Encoding
Length: 196817 (192K) [text/html]
Remote file exists and could contain further links,
but recursion is disabled -- not retrieving.
#


Client SSL Certificate

wget -O index.html  --certificate=OK.crt --private-key=OK.key https://example.com/

wget -O index.html  --certificate=NG.crt --private-key=NG.key https://example.com/








Linux Commands#Network
curl - How to use curl command on Linux