How to test if a URL exists from a .NET program?

Gary,

If you're talking about the same scenario as in your earlier post and if I'm understanding what you're needing then, I believe, screen-scraper could handle this. Are you manually constructing the URL versus extracting the link from the page? Is this perhaps why you're wanting to verify that it is a valid URL before requesting it? If so, then you have to do a bit of dancing just to determine whether or not you'll get a 404 without downloading the entire file.

The script below will first indicate if the URL is valid and if it is return the contentType. The contentType is what your browser looks at to know whether or not to load Acrobat or prompt you to download a file, for example.

It's likely that .Net has a way to do this that may not be so verbose, but if you create a script using the following, pass the script your URL as a session variable and check the value of the session variable "CONTENT_TYPE".

import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.methods.*;
import org.apache.commons.httpclient.params.HttpMethodParams;
import org.apache.commons.httpclient.contrib.ssl.EasySSLProtocolSocketFactory;

session.log("URL: " + session.getVariable( "URL" ));
if (session.getVariable( "URL" )!=null)
{
	session.log("No, not null");
	
	urlString = session.getVariable( "URL" );
	session.setVariable("URL", null);
	
	session.log("Checking content-type for: " + urlString);
	
	// Create a method instance.
	HeadMethod method = new HeadMethod( urlString );
	
	// Provide custom retry handler is necessary
	method.getParams().setParameter
	(
	  HttpMethodParams.RETRY_HANDLER,
	  new DefaultHttpMethodRetryHandler( 3, false )
	);
	
	try
	{
		HttpClient client = new HttpClient();
		
		session.setProxySettingsOnHttpClient( client, client.getHostConfiguration() );
		
		try
		{
			HostConfiguration hostConfiguration = new HostConfiguration();
			URL url = new URL( urlString );
			if( url.toString().startsWith( "https" ) )
			{
				Protocol easyHTTPS = new Protocol( "https", new EasySSLProtocolSocketFactory(), 443 );
				hostConfiguration.setHost( url.getHost(), 443, easyHTTPS );
			}
			else
			{
				hostConfiguration.setHost( url.getHost() );
			}
		}
		catch( MalformedURLException mfue )
		{
			session.log( "MalformedURLException: " + mfue, mfue );
		}
		
		// Execute the method.
		int statusCode = client.executeMethod( method );
		
		if( statusCode!=HttpStatus.SC_OK )
		{
			throw new Exception( "Received status code: " + statusCode );
		}
		
		// Retrieve just the last modified header value.
		String contentType = method.getResponseHeader( "Content-Type" ).getValue();
	
		session.log( "contentType: " + contentType );
		session.setVariable( "CONTENT_TYPE", contentType );
	}
	catch( Exception e )
	{
		throw e;
	}
	finally
	{
	  // Release the connection.
	  method.releaseConnection();
	}
}