Deft Code: Duplicate Class Path Entries

October 27, 2003

Duplicate Class Path Entries

This is my first entry in what I hope will be an ongoing series called ‘Journey to a Solution’. Basically, I’ll document how I fixed a particular problem. Often when I’m reading how a developer fixed an issue, I’m left wondering how they ever fixed the problem; we all have a tendency to leave out all the dead ends and in-retrospect-dumb things we tried that could never have worked.

So on with the show:

The problem

We’re considering upgrading to Java 1.4.2 from 1.4.0 on our production servers. Taking the bold step of trying this out in development first uncovered a weird issue: whenever running code with 1.4.2 (whether via Tomcat or junit tests w/ant) we got these errors:

    [junit] Oct 27, 2003 5:08:49 PM java.util.jar.Attributes read
    [junit] WARNING: Duplicate name in Manifest: Class-Path

How I fixed it

google search: nada.

Does this mean we have duplicate jars in the classpath? Use ant -verbose test to find out. (Oi, 77 entries. I hate classpath issues.) Nothing obvious, but this doesn’t stop me fiddling fruitlessly with build.xml to no effect.

What the hell is a manifest file again? Google provides. A manifest file is created whenever you create a JAR file. Hey, maybe we have a Manifest file in one of our JARs with a duplicate Class-Path entry. (It should be obvious at this point why I get paid the big bucks.)

So we have somewhere around 70 jar files that could have an extra Class-Path line. (I hate classpath issues.)

We need to: unpack each one, grep for Class-Path, repeat.

I copy all suspect JARs to a temp dir and start doing this. Eventually I remember that computers are good at automating repetitive tasks (who knew?) and write a quick shell script:
```
#!/bin/tcsh

jar xf $1
chmod +x META-INF
grep -i class META-INF/MANIFEST.MF
echo "Finished looking at $1"
rm $1
rm -r META-INF/
ls
```

Now I can quickly scan a particular jar with quick.sh foo.jar. (Full disclosure aside: this is just the final quick.sh; the chmod line is there because I was initially using unzip -l instead of jar xf and it would sometimes not have execute perms on the dirs. As it is now, I don’t think it’s necessary.)

Eventually, I find the cuplrit: struts.jar.

Great! Now I’ve identified the problem, I just need to fix it. This involves unpacking struts.jar, fixing the manifest file (instead of multiple Class-Path: lines with one jar per line, one Class-Path line with multiple jars separated by spaces), and repacking the jar ( jar cmf ../MANIFEST.MF struts.jar *). Test runs now have regular output. Joy.

What I’d do differently next time

Read the error message more carefully. Went a bit down the wrong road because of this.
I haven’t checked this, but I suspect if I read through the release notes for 1.4.2 (and maybe 1.4.1) I’ll find reference to this change.
I suspect if I’d stopped and thought a bit, I could have come up with an even cleaner solution to looking through all those jar files; it could even be that there’s a tool out there for doing just that. If I had greater facility with shell scripts, I could also have done it that way.

Update: Did some browsing of sun’s Java forums, which are unfortunately behind a registration wall, and therefore not indexed by search engines. [That this is stupid is fairly evident, so I won’t comment further upon it.] What I learned is:

The specification is clear that if there is more than one Class-Path entry, only the last will be used. Unfortunately, there is a tutorial somewhere on sun.com that says otherwise. This has resulted in many a bug.
There is also a 72 byte limit to manifest file lines. You can use SPACE + NEWLINE as a continuation marker (again, this is in the spec.)
There is a bug (registration required) that documents the change. This bug is still marked as in progress at this time, although the description indicates it’s a 1.4.2 change
Nothing in 1.4.2’s changes file or README document this change.

Posted by Bill Stilwell at October 27, 2003 06:01 PM

Comments

Beautiful. Why the hell arent all technical documents and tips like this?

Posted by: Ian at November 24, 2003 11:52 AM

Excellent work. I had the same problem, never thought to check the struts jar. Nice description. Thanks!

Posted by: Mark at November 25, 2003 07:22 PM

Very helpful information. Nice job and thank you.

Posted by: Geoff at January 6, 2004 10:15 AM

Nice. Was wondering what the issue was.

Since Struts is headed by a person who works for Sun, you wouldn't expect this.

Thanks!

Posted by: Sloan at January 11, 2004 06:39 PM

Great. I have been struggling with the same problem for months.

Thank you!

Posted by: Ville at January 28, 2004 02:35 PM

I am having the same problem, and can't seem to find the duplicate class-path anywhere. However you might find updated script this useful.

It is much faster because it doesn't unjar the whole jarfile, just the manifest.

If is safe to use in any lib directory - because if moves the current META-INF out of the way, and then puts it back at the end. (If you stop in the middle, the directory will be named META-INF.(process number)

It automatically checks all .jar files in the current directory. It does use bash, but should be easy to adapt to your shell of choice.

#!/bin/bash

if [[ -d META-INF ]] ; then mv META-INF META-INF.$$; fi

for file in *.jar
do
echo "Checking $file"
jar xf $file META-INF/MANIFEST.MF
if [[ -a META-INF/MANIFEST.MF ]]
then
chmod +x META-INF
grep -i class-path META-INF/MANIFEST.MF
rm -r META-INF/
else
echo "no manifest for $file"
fi
done

if [[ -d META-INF.$$ ]]; then mv META-INF.$$ META-INF; fi

Posted by: Rich at February 27, 2004 10:40 PM

Hmm, do you have any packaged war files that might have jars in them? The only thing I can think of off the top of my head.

Thanks for the script! Much better than mine.

Posted by: Bill Stilwell at February 27, 2004 11:57 PM

Smart, Real Smart reading and very helpful. Who would have imagined software engineers had the humor bone. Thank you for documenting your effort and saving me some headache from those annoying warnings.

Posted by: John Smith at March 8, 2004 02:06 PM

I think I have a similar problem (Windows XP, Ant, JRE)

Buildfile: build.xml

clean:
[delete] Deleting directory C:\DOCUME~1\GJ5A2E~1.DEG\LOCALS~1\Temp\G.J. de Graaf
[delete] Deleting directory D:\apache-ant-1.6.1\bin\build

genResumeTag:

init:
[mkdir] Created dir: D:\apache-ant-1.6.1\bin\build
[copy] Copying 1 file to D:\apache-ant-1.6.1\bin\build
[filter] Reading filters from D:\apache-ant-1.6.1\bin\build.properties

[echo] Building resume @ 2004-03-24T0247
[echo] user.name: G.J. de Graaf
[echo] os.name: Windows XP
[echo] temp.dir: C:\DOCUME~1\GJ5A2E~1.DEG\LOCALS~1\Temp\

fetch:

xml2fo:
[mkdir] Created dir: C:\DOCUME~1\GJ5A2E~1.DEG\LOCALS~1\Temp\G.J. de Graaf\dist
[echo] generating C:\DOCUME~1\GJ5A2E~1.DEG\LOCALS~1\Temp\/G.J. de Graaf/dist/G.J. de Graaf-resume-2004-03-24T0247.fo

xml2rtf:
[echo] generating C:\DOCUME~1\GJ5A2E~1.DEG\LOCALS~1\Temp\/G.J. de Graaf/dist/G.J. de Graaf-resume-2004-03-24T0247.rtf
[java] 24-mrt-2004 14:47:07 java.util.jar.Attributes read
[java] WARNING: Duplicate name in Manifest: Main-Class
[java] 24-mrt-2004 14:47:07 java.util.jar.Attributes read
[java] WARNING: Duplicate name in Manifest: Main-Class
[java] 24-mrt-2004 14:47:07 java.util.jar.Attributes read
[java] WARNING: Duplicate name in Manifest: Main-Class
[java] 24-mrt-2004 14:47:07 java.util.jar.Attributes read
[java] WARNING: Duplicate name in Manifest: Main-Class
[java] 24-mrt-2004 14:47:07 java.util.jar.Attributes read
[java] WARNING: Duplicate name in Manifest: Main-Class
[java] 24-mrt-2004 14:47:07 java.util.jar.Attributes read
[java] WARNING: Duplicate name in Manifest: Main-Class
[java] 24-mrt-2004 14:47:07 java.util.jar.Attributes read
[java] WARNING: Duplicate name in Manifest: Main-Class

Can you help????
Thanks,
George

Posted by: George at March 24, 2004 07:32 AM

I think I have a similar problem (Windows XP, Ant, JRE)

Buildfile: build.xml

clean:
[delete] Deleting directory C:\DOCUME~1\GJ5A2E~1.DEG\LOCALS~1\Temp\G.J. de Graaf
[delete] Deleting directory D:\apache-ant-1.6.1\bin\build

genResumeTag:

init:
[mkdir] Created dir: D:\apache-ant-1.6.1\bin\build
[copy] Copying 1 file to D:\apache-ant-1.6.1\bin\build
[filter] Reading filters from D:\apache-ant-1.6.1\bin\build.properties

[echo] Building resume @ 2004-03-24T0247
[echo] user.name: G.J. de Graaf
[echo] os.name: Windows XP
[echo] temp.dir: C:\DOCUME~1\GJ5A2E~1.DEG\LOCALS~1\Temp\

fetch:

Can you help????
Thanks,
George

Posted by: George at March 24, 2004 07:33 AM

I'm not sure if you are aware of this or not, but there are reported bugs in ANT that was causing this duplicate class-path problem. These bug ID's in ANT's bug tracker describe the problem and show that they have been fixed (18864, 20004, 24702).

ANT 1.6.1 has this problem corrected.

Just thought you might want to know.

I'm glad you posted this info on your blog. It made finding the solution to the problem much easier. :)

Posted by: Dave at April 1, 2004 08:30 AM

Thanks for the help!

Posted by: Steve at April 29, 2004 05:30 AM

Just wondering if you could help me out here. I'm getting a java error, it's with the same log file as the "WARNING: Duplicate name in Manifest"

A little further down it's saying this...

Exception occurred during event dispatching:

java.lang.IllegalStateException: Can't dispose InputContext while it's active

at sun.awt.im.InputContext.dispose(Unknown Source)

at java.awt.Window$1DisposeAction.run(Unknown Source)

at java.awt.Window.dispose(Unknown Source)

at java.awt.Dialog.disposeImpl(Unknown Source)...

any ideas that may point me in the right direction?

Thanks!

Posted by: Whitney at August 5, 2004 09:33 AM

news

Posted by: news- at August 8, 2004 09:52 AM

Thnx a LOT! :)

Posted by: Ragnhild at August 27, 2004 01:52 AM