Get html file Java

Question

Duplicate:

How do you Programmatically Download a Webpage in Java?

How to fetch html in Java

I\'m developping an application th

Answer 1

Funnily enough I wrote utility method that does just that the other week

/**
 * Retrieves the file specified by fileUrl and writes it to 
 * out.
 * 

 * Does not close out, but does flush.
 * @param fileUrl The URL of the file.
 * @param out An output stream to capture the contents of the file
 * @param batchWriteSize The number of bytes to write to out
 *                       at once (larger files than this will be written
 *                       in several batches)
 * @throws IOException If call to web server fails
 * @throws FileNotFoundException If the call to the web server does not
 *                               return status code 200. 
 */
public static void getFileStream(String fileURL, OutputStream out, int batchWriteSize)
                            throws IOException{
    GetMethod get = new GetMethod(fileURL);
    HttpClient client = new HttpClient();
    HttpClientParams params = client.getParams();
    params.setSoTimeout(2000);
    client.setParams(params);
    try {
        client.executeMethod(get);
    } catch(ConnectException e){
        // Add some context to the exception and rethrow
        throw new IOException("ConnectionException trying to GET " + 
                fileURL,e);
    }

    if(get.getStatusCode()!=200){
        throw new FileNotFoundException(
                "Server returned " + get.getStatusCode());
    }

    // Get the input stream
    BufferedInputStream bis = 
        new BufferedInputStream(get.getResponseBodyAsStream());

    // Read the file and stream it out
    byte[] b = new byte[batchWriteSize];
    int bytesRead = bis.read(b,0,batchWriteSize);
    long bytesTotal = 0;
    while(bytesRead!=-1) {
        bytesTotal += bytesRead;
        out.write(b, 0, bytesRead);
        bytesRead = bis.read(b,0,batchWriteSize);;
    } 
    bis.close(); // Release the input stream.
    out.flush();        
}

Uses Apache Commons library i.e.

import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.params.HttpClientParams;