Extra TL;DR When creating zip archives with SharpZipLib make sure you set the file size, disabling Zip64 is neither here nor there.

A project I am working on sources a zip archive from a Web service, extracts the XML file from the zip and then does silly amounts of processing of the data in that XML to produce a new XML file which is returned to the user.

But the sourced zip archive cannot be opened using Python’s zipfile module, and when saved on a Mac the archive cannot be opened using the built-in Archive Utility.app. If one double-clicks the zip, Archive Utility.app just compresses it again and sticks “.cpgz” on the end of the file name.

Fortunately the developer of the Web service is very helpful (the service is written in C# and runs on Windows) and although he didn’t know why Macs were having problems (the built-in Windows zip tool can handle the archive fine) he showed me the code that creates the zip file.

Turns out they were using SharpZipLib, an open-source library for C#. And it turns out SharpZipLib creates archives using Zip64 format by default.

The fix was to disable Zip64 when creating the archive. Here’s a trivial command-line program that creates a zip and disables Zip64 for Mac compatibility:

using System;
using System.IO;
using ICSharpCode.SharpZipLib.Core;
using ICSharpCode.SharpZipLib.Zip;


public class ZipTool
{
    public static void Main(string[] args)
    {
        if (args.Length != 2) {
            Console.WriteLine("Usage: ziptool <input file> <output file>");
            return;
        }

        using (ZipOutputStream zipout = new ZipOutputStream(File.Create(args[1]))) {
            byte[] buffer = new byte[4096];
            string filename = args[0];

            zipout.SetLevel(9);

            // Disable Zip64 for Mac compatibility
            zipout.UseZip64 = UseZip64.Off;

            ZipEntry entry = new ZipEntry(Path.GetFileName(filename));            
            entry.DateTime = File.GetLastWriteTime(filename);
            zipout.PutNextEntry(entry);

            using (FileStream fs = File.OpenRead(filename)) {
                int sourceBytes;
                do {
                    sourceBytes = fs.Read(buffer, 0, buffer.Length);
                    zipout.Write(buffer, 0, sourceBytes);
                } while (sourceBytes > 0);
            }

            zipout.Finish();
            zipout.Close();
        }
    }
}

The disadvantage of disabling Zip64 is you cannot create archives larger than 4 gigabytes, nor can you add files larger than 4 gigabytes before compression.

The advantage of disabling Zip64 is you make me and all my Mac-using hippy friends happy. In concrete terms, disabling Zip64 makes it more likely I will buy you a drink.

Hi Gaston!

Also many thanks to the maintainer of NAnt in MacPorts, who responded to my bug report and pushed an updated NAnt to MacPorts extremely quickly. Although SharpZipLib doesn’t officially support Mono, it builds without a hitch using the “build-net-2.0″ target if you hack the SharpZlib.build NAnt script like so:

<target name="build-mono-2.0" >
    <call target="build-net-2.0" />
    <!--<fail message="Dont know how to build for Mono 2.0." />-->
</target>

Also thanks to the Mono project! Saved me having to fire up a Windows virtual machine to figure out this problem.

The next problem was that MacPorts couldn’t update because rsync was blocked by the corporate access policy. Fortunately plain HTTP is permitted outbound. Here’s how to use a local ports tree.

Install MacPorts using the disk image for 10.5.

curl -O http://distfiles.macports.org/MacPorts/MacPorts-1.8.2-10.5-Leopard.dmg
hdiutil attach MacPorts-1.8.2-10.5-Leopard.dmg
sudo installer -pkg /Volumes/MacPorts-1.8.2/MacPorts-1.8.2.pkg -target /
hdiutil detach /Volumes/MacPorts-1.8.2

If the MacPorts install directories are not in your $PATH environment, you can add them to your .profile. This change will not take effect until you start a new terminal session.

cat >> ~/.profile <<EOF
PATH=/opt/local/bin:/opt/local/sbin:${PATH}
MANPATH=/opt/local/share/man:${MANPATH}
EOF

After you have installed MacPorts, create a directory for the ports tree and check it out using Subversion.

sudo mkdir -p /opt/local/var/macports/sources/svn.macports.org/trunk/dports
cd /opt/local/var/macports/sources/svn.macports.org/trunk/dports
sudo svn co http://svn.macports.org/repository/macports/trunk/dports/ .

N.B. In the last line beginning svn co ... the trailing directory separator is significant!

Now tell MacPorts to use the local checkout rather than rsync. Edit /opt/local/etc/macports/sources.conf and add a new line to the end with the path to the ports tree, then comment out the previous line that uses rsync. Here are the last lines from my configuration:

#rsync://rsync.macports.org/release/ports/ [default]
file:///opt/local/var/macports/sources/svn.macports.org/trunk/dports/ [default]

Finally you must create an index for the tree (otherwise you will see messages saying “Warning: No index(es) found!”).

cd /opt/local/var/macports/sources/svn.macports.org/trunk/dports
sudo portindex

Now go do great things.