Question : .net httpwebrequest. Need to load full web page not just html

My code below is not loading the images from the web page. I am using Firefox with Yslow, and Yslow indicates the page size with images is 251kb. The pure html contact matches the result from my doc.length statement. So my pull is not a complete web page load, rather only the html. Within the html I of course have
How can I in vb.net download a full webpage, images, java, css everything ? My little program is timing the download and sending an alert of the page load time is greater than 8 seconds. Without the graphics my page load times are under 1/2 second. Add the 200kb header image plus other graphic content and my yslow time is 3-4 seconds.

Code Snippet:

         
           
             1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:

           
           
             Do While loopctr < 1000
            Try
                Dim req1 As HttpWebRequest = HttpWebRequest.Create(site)
            Catch e2 As UriFormatException
                lblbox.Text = e2.Message.ToString
                Exit Sub
            End Try
            Dim tickStart1 As Integer = Environment.TickCount
            Dim req As HttpWebRequest = HttpWebRequest.Create(site)
            Dim s As Stream
            Dim resp As WebResponse
            req.Accept = "image/gif, image/jpeg, image/pjpeg, text/plain, text/html, */*"
            req.ContentType = "image/gif, image/jpeg, image/pjpeg, text/plain, text/html, */*"
 
            Try
                req.Timeout = 10000
                resp = req.GetResponse()
            Catch e As WebException
                TextBox4.Text = e.Message
                errcounter = errcounter + 1
                TextBox1.Text = errcounter
                GoTo exitloop
            End Try
            TextBox3.Text = resp.ResponseUri.ToString
            TextBox3.Text = TextBox3.Text & "::" & resp.IsFromCache
            Try
                s = resp.GetResponseStream()
            Catch e As Exception
                TextBox4.Text = e.ToString
                errcounter = errcounter + 1
                TextBox1.Text = errcounter
            End Try
            Dim sr As StreamReader = New StreamReader(s, Encoding.ASCII)
            Dim doc As String
            Try
                doc = sr.ReadToEnd()
            Catch e As WebException
                TextBox4.Text = e.ToString
                errcounter = errcounter + 1
                TextBox1.Text = errcounter
            End Try
            Dim contentlength As Integer = doc.Length

           
         

Open in New Window

Select All

Answer : .net httpwebrequest. Need to load full web page not just html

The only way to do what you want would be to mimic what the browser does. Once you've got the HTML, parse through it looking for anything that references something back on the server (i.e. img, link, iframe...) and request those resources.

A browser doesn't "download a full webpage" either, it does exactly what I described above (try using Fiddler and you'll see dozens of requests when you load a web page while it asks for each resources in turn).