Problem with url.openStream() - too many connections
807589Jul 2 2008 — edited Jul 8 2008I have code that retrieves thousands of files from an http server but I'm getting this error "java.net.BindException: Address already in use: connect".
Below is sample source code and error when run on Windows XP SP2. I think the error is different on a RHELinux box, but I can't get to that box right now, so I don't have the error (Linux seems to fail much later than Windows). I think the error on Linux was something about too many (open?) connections.
So I'm guessing that my OS isn't releasing the connections quickly enough. The only work around I've been successful at is when the exception occurs, loop and sleep until the exception doesn't occur - a bad kludge and very slow.
Any suggestions? What am I doing wrong?
While it is running, if I run netstat, I get lots of the following:
C:\>netstat
TCP BOB:3518 192.168.0.104:http TIME_WAIT
TCP BOB:3519 192.168.0.104:http TIME_WAIT
....
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
public class URLTest {
public static void main(String[] args) throws IOException {
int cnt = 0;
try {
for (int i = 60000; i > 0; i--) {
cnt++;
//Typically I would get distinct URL files.
//It produces the same error as this test hitting one file
URL url = new URL("http://192.168.0.104/icons/apache_pb2.gif");
InputStream in = url.openStream();
//...do something with the data...
in.close();
//I have tried doing disconnect on the connection.
}
} catch (Exception e) {
System.err.println("***Failed after " + cnt + " attempts.");
e.printStackTrace();
}
System.out.println("Done processing.");
}
}
HERE IS THE ERROR
***Failed after 3948 attempts.
java.net.BindException: Address already in use: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.Socket.connect(Socket.java:516)
at java.net.Socket.connect(Socket.java:466)
at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:365)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:477)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:214)
at sun.net.www.http.HttpClient.New(HttpClient.java:287)
at sun.net.www.http.HttpClient.New(HttpClient.java:299)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:796)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:748)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:673)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:917)
at java.net.URL.openStream(URL.java:1007)
at my.test.url.URLTest.main(URLTest.java:16)
Thanks,
Jim