Skip to Main Content

Java EE (Java Enterprise Edition) General Discussion

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Include External Resource with URLConnection

843836Mar 3 2005 — edited Aug 2 2007
I have a jsp templated site and we want to include a Google search, using their sitesearch option, problem is we want to wrap the google output with our template.

Here's what I have:

index.jsp has a form with an action of search.jsp, and a text input named q:
<form action="/search.jsp" method="GET">
<input type="text" name="q"/>
<input type="submit" value="Search"/>
</form>
Here is search.jsp:
<%@ page language="java" import="java.util.*" %>
  <jsp:include page="/header.jsp">
  <jsp:param name="title" value="Mysite Inc. -- Search Results"/>
  <jsp:param name="group" value=""/>
  </jsp:include>
<%
  String query = "/servlet/search?";
  query += request.getQueryString();
%>
  <jsp:include page="<%=query%>"/>
  <jsp:include page="/footer.jsp"/>
As you can see, this includes a servlet. It reads the query string and appends it to forward to the servlet. Servlet code is:
package com.mysite.search;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.net.URL;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.mysite.basetags.Logger;

public class SearchServlet extends HttpServlet {

	public void doGet(HttpServletRequest request, HttpServletResponse response)
			throws ServletException, IOException {
		String url = "http://www.google.com/search";
		url += "?sitesearch=www.mysite.com&output=googleabout&submit=Search";
		String q = request.getParameter("q");
		url += "&q=" + q;
		URL google = new URL(url);
		try {
			BufferedReader input = new BufferedReader(new InputStreamReader(google.openStream()));
			String line = "";
			PrintWriter out = response.getWriter();
			while ((line = input.readLine()) != null) {
				out.println(line);
			}
			out.flush();
			input.close();
		}
		catch(IOException ex) {
			Logger.logError(request.getSession(), ex);
			try {
				PrintWriter out = response.getWriter();
				out.println("Error loading search page: " + url);
			}
			catch(IOException ioe) {
				Logger.logError(request.getSession(), ioe);
			}
		}
	}
}
As you can see I've included some code to spit out the URL onto the webpage so I could see what was being generated. The generated URL is correct and looks like this if I search for "foam":
http://www.google.com/search?sitesearch=www.mysite.com&output=googleabout&submit=Search&q=foam
So when I run this, the page formats just great, and then instead of printing the expected Google page between the header and footer, it prints the URL (as a result of an IOException). The exception looks like this:
java.io.IOException: Server returned HTTP response code: 403 for URL: http://www.google.com/search?sitesearch=www.mysite.com&output=googleabout&submit=Search&q=foam
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:704)
	at java.net.URL.openStream(URL.java:955)
	at com.lamatek.search.SearchServlet.doGet(SearchServlet.java:24)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:684)
	at org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:575)
	at org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:498)
	at org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:822)
	at org.apache.jsp.search_jsp._jspService(search_jsp.java:50)
	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:137)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:210)
	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
	at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:494)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
	at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
	at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2416)
	at com.gefionsoftware.lws.core.StandardHostValve.invoke(StandardHostValve.java:156)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
	at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
	at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
	at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
	at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:601)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:392)
	at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:572)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:619)
	at java.lang.Thread.run(Thread.java:536)
If I paste the url above into IE (of course my site is not mysite...) it works exactly as expected. Am I missing something?
Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on Aug 30 2007
Added on Mar 3 2005
7 comments
280 views