Question : How to use curl to navigate via web proxy

I am learning curl and I am trying to use curl to simply download a remote webpage (www.msn.com) via a free HTTP web proxy.  I can't seem to get the remote page though using curl.

I am fairly certain that cookies and/or java(script) are not an issue because I disabled both in my browser as a test and I am still able to reach www.msn.com manually using a browser through the proxy.

The code I am currently trying is below.  Does anyone have any suggestions at all regarding what I might be doing wrong?

Thank you very much,
Curt

Code Snippet:

1: 2: 3: 4: 5: 6: 7: 8:

#User Agent String UA=Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.13) Gecko/2009080315 Ubuntu/9.04 (jaunty) Firefox/3.0.13" # first just go to Web Proxy Page curl -L -A "${UA}" http://www.incognitobrowse.info/ > www.incognitobrowse.info.html # will parse later but for now just manually plug input values in here and post curl -L -A "${UA}" --referer "http://www.incognitobrowse.info/;auto" --data-urlencode "q=http://www.msn.com&hl[include_form]=on&hl[accept_cookies]=on&hl[show_images]=on&hl[show_referer]=on&hl[base64_encode]=on&hl[strip_meta]=on&hl[session_cookies]=on" -e "http://www.incognitobrowse.info/" http://www.incognitobrowse.info/index.php > anon_msn.html

Open in New Window

Select All

Answer : How to use curl to navigate via web proxy

I was encountering problems because I was not URLEncoding the post data correctly.  Once I encoded each input individually, it worked.