JINI Service Availability Problems

(or Where's my Service?)

Introduction

Many developers find themselves struggling to get a JINI service available within the lookup service. There are a number of common problems which I will present in this article. I will then describe an approach which you can use to determine exactly which problem you're facing. Hopefully, this information will allow you to locate the issue and fix it.

Problems, Problems...

There are several reasons a service may fail to register with a lookup service:

  1. Multicast network issues - most often causes by the service and Reggie being separated by a firewall that isn't passing multicast or perhaps because multicast isn't enabled on either the machine running Reggie or the machine running the service.
  2. Group configuration issues - where the lookup service has been configured with one set of groups and the service has been configured with a different set of groups with no intersection between the two sets.
  3. Service-side codebase problems - typically caused by the Reggie codebase containing a hostname which is not resolvable by the machine attempting to register its service.
  4. Firewall problems - where access to/from Reggie is blocked in some fashion.

These problems can also prevent a client locating a Reggie instance causing a service lookup to fail. Assuming the client has located the Reggie instance there is one further problem which can occur:

  1. Client-side codebase problems - where the service or Reggie has an incorrectly specified codebase, typically with an unresolvable hostname or an incorrect path.

Two other things that can cause lookup to fail:

  1. Broken proxy references - typically the IP address/hostname embedded in the proxy is determined automatically and, in most cases, it will be correct but not always resulting in address resolution problems.
  2. Incorrect use of the lookup API's.

Which Problem?

Client-side Codebase Access Issues

The client side codebase problem is relatively easy to eliminate. We need to test the validity of the URL's the client must download code from. For this, all that's required is a browser installed on the client machine. For each codebase URL (there should be one string of URL's for Reggie and one for the service) - type it into the browser and verify that URL is valid. In the case of .jar's you should be able to download the .jar. In the case of a directory, you should be able to view the directory hierarchy containing your .class's.

Failure to resolve any of these URL's points to a hostname resolution or incorrect path. If you can ping the hostname, chances are you have an incorrect path specified in the service.

Service-side Codebase Issues

Unlike client-side codebase problems, service-side codebase problems occur only when the Reggie codebase is unresolvable by the service attempting to register. Note that if the service itself has an incorrect codebase, this will only manifest itself at client lookup time. Reggie doesn't access/use a service's codebase.

You should check the Reggie codebase in the same way as you would for the client-side issues, using a browser.

Group Configuration Issues

Before going on to check for multicast problems (see below), you need to ensure that your client/service groups intersect with those defined for your Reggie instance. This should be relatively easy to do - simply check the relevant configuration files. Alternatively, configure your client/service and Reggie to join all groups which can be achieved by passing in or configuring DiscoveryGroupManagement.ALL_GROUPS. If registration/lookup still fails, first ensure that your codebases are valid and only then investigate the possibility of multicast issues.

Multicast Network Issues/Firewall Issues

Multicast network issues will prevent group-based discovery methods from functioning correctly. To determine if this is the issue, configure your client or service to use a LookupLocator rather than group registration. If service registration or client lookup functions correctly, the problem is related to multicast. Check that:

  1. The client/service machine has multicast enabled.
  2. The Reggie machine has multicast enabled.
  3. Multicast packets are visible at each machine's network interfaces - you'll require either a packet sniffer or some other test package such as Marm (available in the JINI tools package).
  4. Multicast is occuring on the correct network interface.

Broken proxy references

The various services in the JINI starter kit determine the default IP address/hostname embedded in the proxy using something like...

InetAddress.getLocalHost();

... from which the hostname is extracted. Reggie also uses the above for such things as it's advertisements. If the resulting hostname is not resolvable via DNS, the local hosts file or some other mechanism, one will encounter connectivity and lookup issues.

The solution is either to configure the chosen service to use IP addresses (via configuration variables in it's .config file or to ensure the hostname is resolvable. In the case of the services in the JINI starter kit, the JavaDoc provides the necessary detail.

Incorrect use of the lookup API's

Dynamic discovery of lookup services and the services they contain is asynchronous by nature and can cause problems for client programmers. Particular attention should be paid to the fact that many of the lookup methods are non-blocking and can fail to return a matching service simply because the infrastructure hadn't been running for long enough to receive advertisements from available lookup services.

In these cases, programmers should either switch to using blocking methods (which often accept a wait duration) or make use of a ServiceDiscoveryListener which can be used to trigger client operations once a suitable service has been discovered asynchronously.

Tools

A list of tools and their uses:

  1. Marm - a Java-based tool for searching for JINI multicast packets.
  2. Tcpdump/snoop - packet sniffers found on most UNIX'en.
  3. Inca X Service Browser - allows you to examine codebase URL's, interfaces of registered proxies, search for lookup services etc. Almost all the above issues can be tested for using this tool.
  4. Logging - both ServiceDiscoveryManager and JoinManager produce significant logging information in respect of their actions which can be exceedingly useful for debugging. See Appendix A for suggestions on use/configuration.

Appendix A

Enabling logging in client or service is achieved by adding this to the JVM command-line:

-Djava.util.logging.config.file=logging.properties

logging.properties should look something like this:

handlers= java.util.logging.ConsoleHandler

.level=ALL

# Limit the message that are printed on the console to INFO and above.
java.util.logging.ConsoleHandler.level = FINEST
java.util.logging.ConsoleHandler.formatter =
java.util.logging.SimpleFormatter


############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################

# For example, set the com.xyz.foo logger to only log SEVERE
# messages:
#net.jini.loader.pref.PreferredClassProvider.level = ALL
#net.jini.jeri.tcp.client.level = FINEST
#net.jini.jeri.BasicInvocationHandler.level = FINEST
#net.jini.jeri.connection.ConnectionManager.level = FINEST
#net.jini.lookup.ServiceDiscoveryManager.level = FINEST
net.jini.lookup.JoinManager = FINEST

The additional commented-out levels may also be useful under some circumstances (see the JINI JavaDoc for more information on these).

Appendix B - A Quick Word on Codebases

There are two basic varieties of codebase URL. The first references in a .jar (for example http://dredd:8080/reggie-dl.jar whilst the second references a tree of .class files arranged in a directory structure to match their packages.

It is this second form that needs a little attention. For whatever reason, the specific structure of this kind of codebase specification is very important. You must ensure that you terminate the URL with a "/" so this...

http://dredd/myclasses/

...is correct whilst this will likely result in code-downloading problems... 

http://dredd/myclasses

© Copyright 2005 Dan Creswell