Error when scanning JAR files containing AppleDouble metadata

[0dinD]

Related to MCRcortex/nekodetector#53

I was scanning some JAR files and noticed that one of them resulted in an error from the detector: Error: Invalid MAGIC value.

• Terralith_v2.2.1a.jar: 12 classes error
  • __MACOSX/net/hypercubemc/seedfix/._SeedFix.class: error
    Error: Invalid MAGIC value
  • __MACOSX/net/hypercubemc/seedfix/mixin/._TheEndBiomeSourceAccessor.class: error
    Error: Invalid MAGIC value
  • __MACOSX/net/hypercubemc/seedfix/mixin/._TheEndBiomeSourceMixin.class: error
    Error: Invalid MAGIC value
  • __MACOSX/net/hypercubemc/seedfix/mixin/._NoiseBasedChunkGeneratorAccessor.class: error
    Error: Invalid MAGIC value
  • __MACOSX/net/hypercubemc/seedfix/mixin/._NoiseBasedChunkGeneratorMixin.class: error
    Error: Invalid MAGIC value

The file in question is this one: https://www.curseforge.com/minecraft/mc-mods/terralith/files/3696607

The Error: Invalid MAGIC value seems to come from here: https://github.com/leonardosnt/java-class-tools/blob/4e632bf8fafa89243c3b0e2786a4fe496864f262/src/java-class-reader.js#L36-L39

Why does this happen?

Because the Terralith JAR (and many other JARs on CurseForge etc) was created on macOS, and was not properly stripped of AppleDouble metadata. For reference, see:

• https://en.wikipedia.org/wiki/AppleSingle_and_AppleDouble_formats

  • Mac OS X revived the use of AppleDouble; on file systems such as NFS and WebDAV that do not natively support resource forks, Finder information, or extended attributes, that information is stored in AppleDouble format, with the second file having a name generated by prepending "._" to the name of the first file (thus, this information acts as a hidden file when viewed from a non-Apple Unix-based operating system). The files are sometimes moved to a separate directory called .AppleDouble. Metadata separation is also done in the OS X 10.3+ Finder Zip compression and the command line ditto utility, with a copy of the AppleDouble metadata stored in a __MACOSX directory.

• https://en.wikipedia.org/wiki/Resource_fork

So those JARs contain .class files which are not real class files, they just contain Apple metadata. Which is why we get an error when trying to scan those class files.

Potential solutions

Perhaps the scanning tool could exclude all files in the __MACOSX directory. But I think that could provide a place for bad actors to hide infected class files, since __MACOSX is a valid Java package name, so excluding that directory seems like a bad idea.

Another idea could be to exclude the metadata files based on the fact that they start with ._. To my knowledge, a valid Java class name can't contain the . character, but I don't want to say that I'm 100% sure what the rules are on a bytecode level.

Yet another idea would be to do some error handling and make sure that the scanning tool still continues to scan all other classes even if it runs into some invalid class files. Maybe it already does this (?), I did not look at the code in detail. But even if the scanning tool already does this, I think it would be a good idea to provide some more helpful information to the end user when they scan a JAR file and it contains invalid class files that lead to errors. Like, what does that mean ("is my JAR file infected or not?"), and what action should the user take?

[douira]

Thanks for the report. I think an easy way to solve this is to listen for this specific error and treat it as "clean".

[0dinD]

As in, ignore any Error: Invalid MAGIC value errors? That sounds like it could work. As far as I understand, if we get that error, then the class file should be invalid to the point where it must be harmless? But again, I'm not 100% sure about the technicalities here.

The main concern I have about ignoring errors from class files is that we don't want to provide a way for bad actors do hide or "disguise" malicious class files by naming them in a certain way or by doing some bytecode trickery.

[douira]

If the magic value is wrong, the file wouldn't be loaded as a class by the JVM as far as I'm aware.

[0dinD]

Yeah, that seems reasonable to assume.

[douira]

The program could of course somehow load this file, modify it in memory, and then read it as a class but detecting that is the job of the signature and out of scope for the detector to do itself.