I'm writing a fairly basic web spider which checks the validity of all the links on a given website. I wanted a class which handled parsing/connection/etc URLs, so I came up with this (incomplete due to the error I'm getting!):
import java.net.*;
import java.io.*;
import java.util.*;
public class URLHandler
{
private URL url;
private URI uri;
private boolean validURLString;
private boolean validURLConnection;
private URLConnection connection;
private BufferedReader stream;
public URLHandler(String url)
{
String protocol;
Properties systemSettings = System.getProperties();
systemSettings.put("http.proxyHost", "");
systemSettings.put("http.proxyPort", "");
System.setProperties(systemSettings);
if (url.indexOf("://") != -1){
protocol = url.substring(0, url.indexOf("://"));
url = url.substring(url.indexOf("://")+3);
} else { // because we're dealing with websites we can assume default is http
protocol = "http";
}
try{
uri = new URI(protocol,url,null);
try{
this.url = uri.toURL();
} catch (MalformedURLException e) {
validURLString=false;
} finally {
validURLString=true;
}
} catch (URISyntaxException e) {
validURLString=false;
}
}
public boolean connectToURL()
{
System.out.println(url.getPath());
try{
connection = this.url.openConnection();
connection.connect();
stream = new BufferedReader( new InputStreamReader (connection.getInputStream()) );
} catch (IOException e){
validURLConnection = false;
} finally {
validURLConnection = true;
}
return validURLConnection;
}
public void write() throws IOException
{
String inputLine;
while ((inputLine = stream.readLine()) != null)
System.out.println(inputLine);
stream.close();
}
}
I've removed the proxy settings, but I do need them as I'm behind my university's firewall. Anyway, when I invoke the connectToURL() method I just get a null pointer exception, and I'm not sure why.
Any help would be greatly appreciated!
Pete
Edited by: Puffy on Nov 13, 2007 9:08 AM