HOW TO: Implement Screen Scraping with ASP.NET

Solution

• Screen Scraping, ViewState, and Authentication using ASP.Net by K. Scott Allen (OdeToCode.com) - examines 3 screen scraping methods: using HttpServerUtility, WebClient, and HttpWebRequest. The article also explains how to deal with retrieving content from a web site requiring forms authentication and viewstate handling.
• Easy .NET Screen Scraping by Steven Smith (ASPAlliance) - besides explaining the basics in implementing screen scraping, the article also explains how to work with international character sets in scraped pages. Part 2 of article: Using POST with Scraping and .NET
• Screen Scrapes in ASP.NET by Scott Mitchell (4GuysFromRolla.com)
• Screen Scraping in ASP.NET by Salman (CSharpFriends.com)
• Screen Scraping with C# for ASP.NET by Mark Nischalke (CodeProject.com)

Related Resources

• MSDN: HttpWebRequest Class
• Book: Chapter 21, ASP.NET Developer's Cookbook by Steven Smith and Rob Howard