Download a file from the internet using java : How to authenticate?

Question

Thanks to this thread How to download and save a file from Internet using Java? I know how to download a file, now my problem is that I need to authenticate on the sever from which I'm dowloading. It's an http interface to a subversion server. Which field do I need to look up into ?

Using the code posted in the last comment, I get this exception:

java.io.IOException: Server returned HTTP response code: 401 for URL: http://myserver/systemc-2.0.1.tgz
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1305)
    at java.net.URL.openStream(URL.java:1009)
    at mypackage.Installer.installSystemc201(Installer.java:29)
    at mypackage.Installer.main(Installer.java:38)

Thanks,

Answer 1

You extend the Authenticator class and register it. The javadocs at the link explain how.

I don't know if this works with the nio method that got the accepted answer to the question, but it for sure works for the old fashioned way that was the answer under that one.

Within the authenticator class implementation, you are probably going to use a PasswordAuthentication and override the getPasswordAuthentication() method of your Authenticator implementation to return it. That will be the class which is passed the user name and password you need.

Per your request, here is some sample code:

public static final String USERNAME_KEY = "username";
public static final String PASSWORD_KEY = "password";
private final PasswordAuthentication authentication;

public MyAuthenticator(Properties properties) {
    String userName = properties.getProperty(USERNAME_KEY);
    String password = properties.getProperty(PASSWORD_KEY);
    if (userName == null || password == null) {
        authentication = null;
    } else {
        authentication = new PasswordAuthentication(userName, password.toCharArray());
    }
}

protected PasswordAuthentication getPasswordAuthentication() {
    return authentication;
}

And you register it in the main method (or somewhere along the line before you call the URL):

Authenticator.setDefault(new MyAuthenticator(properties));

The usage is simple, but I find the API convoluted and kind of backwards for how you typically think about these things. Pretty typical of singleton design.

Answer 2

This is some code I wrote that fetches a website and displays the contents to System.out. It uses Basic authentication:

import java.net.*;
import java.io.*;

public class foo {
    public static void main(String[] args) throws Exception {

   URL yahoo = new URL("http://www.MY_URL.com");

   String passwdstring = "USERNAME:PASSWORD";
   String encoding = new 
          sun.misc.BASE64Encoder().encode(passwdstring.getBytes());

   URLConnection uc = yahoo.openConnection();
   uc.setRequestProperty("Authorization", "Basic " + encoding);

   InputStream content = (InputStream)uc.getInputStream();
   BufferedReader in   =   
            new BufferedReader (new InputStreamReader (content));

   String line;
   while ((line = in.readLine()) != null) {
      System.out.println (line);
   }   

   in.close();
}

Problems with the above code:

1. This code isn't production-ready (but it gets the point across.)

2. The code yields this compiler warning:

foo.java:11: warning: sun.misc.BASE64Encoder is Sun proprietary API and may be removed in a future release
      sun.misc.BASE64Encoder().encode(passwdstring.getBytes());
              ^ 1 warning

One really should use the Authenticator class, but for the life of me, I could not figure out how and I couldn't find any examples either, which just goes to show that the Java people don't actually like it when you use their language to do cool things. :-P

So the above isn't a good solution, but it does work and could easily be modified later.

Answer 3

Write your overriding class for Authenticator:

import java.net.Authenticator;
import java.net.PasswordAuthentication;

public class MyAuthenticator extends Authenticator {  
    private static String username = "";
    private static String password = "";

    protected PasswordAuthentication getPasswordAuthentication() {
        return new PasswordAuthentication (MyAuthenticator.username, 
                MyAuthenticator.password.toCharArray());
    }

    public static void setPasswordAuthentication(String username, String password) {
        MyAuthenticator.username = username;
        MyAuthenticator.password = password;
    }
}

Write your main class:

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.Authenticator;
import java.net.MalformedURLException;
import java.net.URL;

public class MyMain{


    public static void main(String[] args) {
        URL url;
        InputStream is = null;
        BufferedReader br;
        String line;

        // Install Authenticator
        MyAuthenticator.setPasswordAuthentication("Username", "Password");
        Authenticator.setDefault (new MyAuthenticator ());

        try {
            url = new URL("Your_URL_Here");
            is = url.openStream();  // throws an IOException
            br = new BufferedReader(new InputStreamReader(is));
            while ((line = br.readLine()) != null) {
                System.out.println(line);
            }
        } catch (MalformedURLException mue) {
             mue.printStackTrace();
        } catch (IOException ioe) {
             ioe.printStackTrace();
        } finally {
            try {
                if (is != null) is.close();
            } catch (IOException ioe) {
                // nothing to see here
            }
        }

    }

}

Answer 4

Have you tried building your URL in the form http://user:password@domain/path?

Answer 5

I would suggest checking out HttpClient from apache http://hc.apache.org/httpclient-3.x/ it makes downloading/authenticating very easy

Answer 6

This open source library, http://spnego.sourceforge.net, also has some examples on how to use it's SpnegoHttpURLConnection class.

One of the constructors in the class allows you to pass-in a username and password.

Take a look at the class's java doc for the examples.