The better solution is to just set the size of each file you add to the archive. That way you can keep Zip64 compatibility and Mac compatibility.

I owe this solution to the excellent SharpZipLib forum, which covered this problem a while ago, but which I missed when I wrote the earlier blog.

Here’s an updated version of the zip tool in C# that makes Macs happy without annoying anyone else:

using System;
using System.IO;
using ICSharpCode.SharpZipLib.Core;
using ICSharpCode.SharpZipLib.Zip;


public class ZipTool
{
    public static void Main(string[] args)
    {
        if (args.Length != 2) {
            Console.WriteLine("Usage: ziptool <input file> <output file>");
            return;
        }

        using (ZipOutputStream zipout = new ZipOutputStream(File.Create(args[1]))) {
            byte[] buffer = new byte[4096];
            string filename = args[0];

            zipout.SetLevel(9);

            //  Set the size before adding it to the archive, to make your
            //  Mac-loving hippy friends happy.
            ZipEntry entry = new ZipEntry(Path.GetFileName(filename));
            FileInfo info = new FileInfo(filename);
            entry.DateTime = info.LastWriteTime;
            entry.Size = info.Length;
            zipout.PutNextEntry(entry);

            using (FileStream fs = File.OpenRead(filename)) {
                int sourceBytes;
                do {
                    sourceBytes = fs.Read(buffer, 0, buffer.Length);
                    zipout.Write(buffer, 0, sourceBytes);
                } while (sourceBytes > 0);
            }

            zipout.Finish();
            zipout.Close();
        }
    }
}

Extra TL;DR When creating zip archives with SharpZipLib make sure you set the file size, disabling Zip64 is neither here nor there.

A project I am working on sources a zip archive from a Web service, extracts the XML file from the zip and then does silly amounts of processing of the data in that XML to produce a new XML file which is returned to the user.

But the sourced zip archive cannot be opened using Python’s zipfile module, and when saved on a Mac the archive cannot be opened using the built-in Archive Utility.app. If one double-clicks the zip, Archive Utility.app just compresses it again and sticks “.cpgz” on the end of the file name.

Fortunately the developer of the Web service is very helpful (the service is written in C# and runs on Windows) and although he didn’t know why Macs were having problems (the built-in Windows zip tool can handle the archive fine) he showed me the code that creates the zip file.

Turns out they were using SharpZipLib, an open-source library for C#. And it turns out SharpZipLib creates archives using Zip64 format by default.

The fix was to disable Zip64 when creating the archive. Here’s a trivial command-line program that creates a zip and disables Zip64 for Mac compatibility:

using System;
using System.IO;
using ICSharpCode.SharpZipLib.Core;
using ICSharpCode.SharpZipLib.Zip;


public class ZipTool
{
    public static void Main(string[] args)
    {
        if (args.Length != 2) {
            Console.WriteLine("Usage: ziptool <input file> <output file>");
            return;
        }

        using (ZipOutputStream zipout = new ZipOutputStream(File.Create(args[1]))) {
            byte[] buffer = new byte[4096];
            string filename = args[0];

            zipout.SetLevel(9);

            // Disable Zip64 for Mac compatibility
            zipout.UseZip64 = UseZip64.Off;

            ZipEntry entry = new ZipEntry(Path.GetFileName(filename));            
            entry.DateTime = File.GetLastWriteTime(filename);
            zipout.PutNextEntry(entry);

            using (FileStream fs = File.OpenRead(filename)) {
                int sourceBytes;
                do {
                    sourceBytes = fs.Read(buffer, 0, buffer.Length);
                    zipout.Write(buffer, 0, sourceBytes);
                } while (sourceBytes > 0);
            }

            zipout.Finish();
            zipout.Close();
        }
    }
}

The disadvantage of disabling Zip64 is you cannot create archives larger than 4 gigabytes, nor can you add files larger than 4 gigabytes before compression.

The advantage of disabling Zip64 is you make me and all my Mac-using hippy friends happy. In concrete terms, disabling Zip64 makes it more likely I will buy you a drink.

Hi Gaston!

Also many thanks to the maintainer of NAnt in MacPorts, who responded to my bug report and pushed an updated NAnt to MacPorts extremely quickly. Although SharpZipLib doesn’t officially support Mono, it builds without a hitch using the “build-net-2.0″ target if you hack the SharpZlib.build NAnt script like so:

<target name="build-mono-2.0" >
    <call target="build-net-2.0" />
    <!--<fail message="Dont know how to build for Mono 2.0." />-->
</target>

Also thanks to the Mono project! Saved me having to fire up a Windows virtual machine to figure out this problem.

Let’s create a new Excel workbook and add a sheet:

>>> import xlwt
>>> book = xlwt.Workbook(encoding='utf-8')
>>> sheet = book.add_sheet('sheeeeeet')

We need to get a column in order to set its width. You do that by call col() on the sheet, passing the column’s index as the only argument (or row() for accessing rows):

>>> sheet.col(0)    # First column
<xlwt.Column.Column object at 0x10b2a6190>
>>> sheet.row(2)    # Third row
<xlwt.Row.Row object at 0x10b2a7050>

The index is zero-based. You can fetch a column even if you have not written to any cell in that column (this applies equally to rows).

Columns have a property for setting the width. The value is an integer specifying the size measured in 1/256 of the width of the character ’0′ as it appears in the sheet’s default font. xlwt creates columns with a default width of 2962, roughly equivalent to 11 characters wide.

>>> first_col = sheet.col(0)
>>> first_col.width = 256 * 20              # 20 characters wide (-ish)
>>> first_col.width
5120

For rows, the height is determined by the style applied to the row or any cell in the row. (In fact rows also have a property called height but it doesn’t do what you want.) To set the height of the row itself, create a new style with a font height:

>>> tall_style = xlwt.easyxf('font:height 720;') # 36pt
>>> first_row = sheet.row(0)
>>> first_row.set_style(tall_style)

Setting the style on the row does not change the style of the cells in that row.

There is no obvious way to set a default width and height for all columns and rows. An instance of xlwt.Worksheet.Worksheet has properties for col_default_width and row_default_height but changing those does not actually change the defaults.

The problem is that new columns are always created with an explicit width, while rows take their height from the style information.

My first attempt at setting defaults set the width on every column and the height on every row. It works, but creates 65,536 unnecessary empty row objects.

A slightly better approach is to set the width on every column and to set the font height on the default style record in the workbook:

import itertools
import xlwt

book = xlwt.Workbook(encoding='utf-8')
sheet = book.add_sheet('sheeeeeet')

col_width = 256 * 20                        # 20 characters wide

try:
    for i in itertools.count():
        sheet.col(i).width = col_width
except ValueError:
    pass

default_book_style = book.default_style
default_book_style.font.height = 20 * 36    # 36pt

book.save('example.xls')

Here I used itertools.count() and wrapped the loop in a try block so I can forget exactly how many columns are permitted. When the loop tries to access a bad index it will throw ValueError and the loop will exit.

You mustn’t replace the default style on an instance of xlwt.Workbook.Workbook, you have to update the property of the existing style (to ensure you are changing the first style record). Unfortunately there is no way to set a default column width (as of xlwt version 0.7.2) so the brute force method of setting every column will have to do – it isn’t so bad since there are only 256 columns.

Talking of widths and heights, have you heard “Widths & Heights” by Magic Arm? Is good.

(This post describes a problem using Adobe InDesign CS4 but applies just as well to CS5 and CS5 and a half.)

Suppose you have a text frame containing the file modification date variable, created using Type → Text Variables → Insert Variable → Modification Date, displays as “14 September 2011 7:50 PM” (on my system). Now open an existing document that was created on a different system and copy and paste the text frame containing the date variable. But depending on what system created the other document the date displays using a different format, e.g as “September 14, 2011 19:50″.

It appears that the date format is determined by the document in which it is placed, rather than by the date format in use when the variable was created.

Workaround: define a new / custom text variable that uses the file modification time but with your own explicit format, then insert your custom variable instead of the pre-defined “Modification Date” variable.

In my brief testing this custom variable and its format is preserved when pasting the text frame into other documents that were created on other systems.

But now I want to know exactly how the format is chosen for the built-in “Modification Date” variable. I am guessing that when a story with a variable is pasted into a document the format is determined by the format of an existing variable of the same name, and if there is no existing variable of that name then InDesign brings in the new variable definition (along with its format) from the clipboard.

But why models?

No, that’s not what I mean… why the default date format? I tried changing the date formats in System Preferences. Doesn’t seem that InDesign picks it up from there. I tried trashing the Adobe InDesign preferences folder, changing the date format in System Preferences and launching InDesign again. InDesign is still using the original format, so doesn’t get it from the user’s preferences. I had a look through ~/Library/Preferences/com.adobe.InDesign.plist but nothing date-related in there.

Perhaps it is set by the local-domain /Library/Preferences/*. Perhaps it is set by the built-in preferences of your installed language version of Creative Suite. Perhaps…

So I gave up. I will leave the investigation for some day when I am younger and it is more important to understand how Adobe’s InDesign picks the format for the built-in date modified variable. The workaround works around.

Suite!

A few weeks ago I received an e-mail from Timothy Williams with changes to the my module so that it preserves the fractions of a second in the conversion. How sweet is that?!?!! Exclamation mark question mark exclamation mark cellida diaresis em-dash full stop king of punctuation.

It is fantastic that not only did someone find my code useful but also that they were generous enough to take the time to improve it and give the changes back to me. I love tasty, delicious free software and the people like Tim who make it tastier and more delicious.

So here is the new version of filetimes.py incorporating Tim’s fixes.

The XPath 1.0 specification says '.' is short-hand for 'self::node()', selecting a node itself.

Parsing an XML document and selecting the context node with ElementTree in Python 2.5:

>>> from xml.etree import ElementTree
>>> ElementTree.VERSION
'1.2.6'
>>> doc = "<Root><Example>BUG</Example></Root>"
>>> node1 = ElementTree.fromstring(doc).find('./Example')
>>> node1
<Element Example at 10e0ed8c0>
>>> node1.find('.')
<Element Example at 10e0ed8c0>
>>> node1.find('.') == node1
True

See how the result of node1.find('.') is the node itself? As it should be.

Parsing an XML document and selecting the context node with cElementTree in Python 2.5:

>>> from xml.etree import cElementTree
>>> doc = "<Root><Example>BUG</Example></Root>"
>>> node2 = cElementTree.fromstring(doc).find('./Example')
>>> node2
<Element 'Example' at 0x10e0e3660>
>>> node2.find('.')
>>> node2.find('.') == node2
False

Balls. The result of node2.find('.') is None.

However! I have a kludgey work-around that works whether you use ElementTree or cElementTree. Use './' instead of '.':

>>> node1.find('./')
<Element Example at 10e0ed8c0>
>>> node1.find('./') == node1
True
>>> node2.find('./')
<Element 'Example' at 0x10e0e3660>
>>> node2.find('./') == node2
True

Kludgey because './' is not a valid XPath expression.

So we are back on track. Also works for Python 2.6 which has the same version of ElementTree.

Fortunately Python 2.7 got a new version of ElementTree and the bug is fixed:

>>> from xml.etree import ElementTree
>>> ElementTree.VERSION
'1.3.0'
>>> doc = "<Root><Example>BUG</Example></Root>"
>>> node3 = ElementTree.fromstring(doc).find('./Example')
>>> node3
<Element 'Example' at 0x107257210>
>>> node3.find('.')
<Element 'Example' at 0x107257210>
>>> node3.find('.') == node3
True

However! They also fixed my kludgey work-around:

>>> node3.find('./')
>>> node3.find('./') == node3
False

So I can’t code something that works for all three versions. This is annoying. I was hoping to just replace ElementTree with the C version, makes my code run in one third the time (the XML parts of it run in one tenth the time). And cannot install any compiled modules – the code can only rely on Python 2.5′s standard library.

Note how none of the files in that list has a name containing the text “david buxton”.

Perhaps deleting the Spotlight index would fix things. Meh.

Any by spreadsheet data I mean CSV is a terrible format for text data that will be opened in Microsoft Excel (unless you only speak English).

My objection to CSV as a spreadsheet format is it has no way to indicate the text encoding, and Excel (at least Excel 2008 for Mac and Excel 2007 for Windows) has no way of choosing the encoding when opening a CSV file. When your data is text from any number of non-English languages you will likely be dealing with characters outside the standard ‘latin-1′ character set.

Even if Excel did have a little pop-up menu for choosing the encoding when opening a CSV file, it would mean instructing your Excel-happy friends to “open as UTF-8″ whenever they wanted to open your CSV data.

Wot you mean you don’t have Excel-happy friends?

To apply formatting to a cell you pass an instance of the xlwt.XFStyle class as the fourth argument to the xlwt.Worksheet.write method. The best way to create an instance is to use the xlwt.easyxf helper, which takes a string that specifies the formatting for a cell.

The other thing about using styles is you should only make one instance of each, then pass that same style object every time you want to apply it to a cell.

An example which uses a few styles:

import xlwt

styles = dict(
    bold = 'font: bold 1',
    italic = 'font: italic 1',
    # Wrap text in the cell
    wrap_bold = 'font: bold 1; align: wrap 1;',
    # White text on a blue background
    reversed = 'pattern: pattern solid, fore_color blue; font: color white;',
    # Light orange checkered background
    light_orange_bg = 'pattern: pattern fine_dots, fore_color white, back_color orange;',
    # Heavy borders
    bordered = 'border: top thick, right thick, bottom thick, left thick;',
    # 16 pt red text
    big_red = 'font: height 320, color red;',
)

I have no idea what it is based on, but 20 = 1 pt. So 320 = 16 pt text.

book = xlwt.Workbook()
sheet = book.add_sheet('Style demo')

for idx, k in enumerate(sorted(styles)):
    style = xlwt.easyxf(styles[k])
    sheet.write(idx, 0, k)
    sheet.write(idx, 1, styles[k], style)

book.save('Example.xls')

It isn’t included with the current distribution on the cheese shop, but there is a useful Excel spreadsheet demonstrating cell patterns in the source repository.

You can find the complete list of possible cell formats by reading the source for xlwt.Styles.

Anyway, for the first few chapters the book leads you through a number of small applications, each exercise starting a new project. Then in the later chapters Hillegass has you adding features to a single project called RaiseMan.

It strikes me that the next edition of the book should introduce using version control to track development.

Many chapters in the book end with challenges that go off on a tangent, getting you to implement alternate approaches to the exercises covered in that chapter. But then the subsequent chapter will expect you to work with the version of the RaiseMan application as it stood before the previous chapter’s programming challenge.

This is a natural fit for revision control (aka VCS). The book would teach you how to tag “release” versions of your code. It would teach you how to create an experimental feature branch for the chapter-end challenges, and then how to resume development from the last “release” version for the next chapter’s main exercises.

This is particularly relevant now that Xcode 4 includes Git integration. Of course there is no reason one can’t employ a suitable version control system with the current (third) edition of the book; I just think it makes a lot of sense to get younglings used to this workflow while teaching them all the other stuff about lightsabres and the Force while you’re at it.

You will up-vote this on Reddit / Hacker News (waves hand like her Imperial majesty).

• An RSS / Atom feed of recent updates.

  This way you can subscribe to a list of recent updates in your favourite news reader and be informed when a new update is released without having to scan product-specific blogs, etc.

• Per-product pages listing available updates.

  When you want to find an update for a product you can find it on the dedicated product update page. It will be there.

• Nice URLs.

  URLs make the Web. Putting the product name in the URL makes a human-friendly URL, as opposed to putting the opaque product reference numeric key as a query parameter. Quick test: which updates do you expect to see on http://www.adobe.com/support/downloads/product.jsp?product=1&platform=Macintosh ? And assuming you were so insane as to guess the previous URL correctly, which updates do you expect to see on http://www.adobe.com/support/downloads/product.jsp?product=1

Anyway, those be my principal beefs with the current Adobe software updates site. So I made a site that tries to satisfy my beeves. Beefs.

It is here: http://reliablybroken.com/wavesinspace/

Please provide feedback to david@gasmark6.com

Adobe’s Downloads page is mostly about downloading product demos. Although on that page there is a list on the side with a link to the real product updates page and a very out-dated list of updates.

So then the actual Product Updates page has a menu for all their products that takes you to the updates for an individual product, and a list of “featured updates”. What qualifies an update to be featured is a mystery, so that list is not useful either.

They don’t think to mention it on the Downloads or Product Updates pages, but there is also a New Downloads page which is actually rather handy, although there is no indication what constitutes “new” so it can be difficult to tell if something was released in the time between your last visit and the oldest update mentioned on that page.

My favourite aspect of Adobe’s support pages is the whimsical approach to the page for a product. For example, the page for Illustrator for Macintosh includes the 15.0.2 update for Illustrator that shipped as part of Creative Suite version 5. Meanwhile the Creative Suite for Mac updates page doesn’t admit there have been any updates for CS5 at all.

What Adobe’s software update site needs is…

UPDATED: fixed ‘build-essential’ – thank you Carlos

But it is easy enough to build a deb from the Perusio package which uses the stable libraries. Here are my notes. N.B. Editing the apt sources and installing packages needs root privileges.

First, add the Perusio repository to /etc/apt/sources.list:

cat >> /etc/apt/sources.list <<EOF
deb http://debian.perusio.net unstable/
deb-src http://debian.perusio.net unstable/
EOF

Update the apt-get index:

apt-get update

Install packaging tools (if they aren’t installed already):

apt-get install dpkg-dev
apt-get install fakeroot
apt-get install debhelper
apt-get install build-essential

Install headers for libaries required to build Nginx:

apt-get install autotools-dev libgeoip-dev libssl-dev libpcre3-dev zlib1g-dev

Now get the package source and build a deb installation package. This part can be done as a non-root user.

apt-get source nginx
cd nginx-0.9.5
dpkg-buildpackage -rfakeroot -uc -b

When I did this, the version in the repository was Nginx 0.9.5. The flag -uc says to not sign the changes file and -b says build a binary-only distribution. If you were cross-compiling, you can pass the -a flag to dpkg-buildpackage specifying the build architecture.

Assuming the build was successful, your new installation package will have been created in the parent directory (bloody hell Debian tools are unintuitive).

Here I built the deb on amd64. To install this I go:

dpkg -i ../nginx_0.9.5-perusio.1.0_amd64.deb

The advantage of building Nginx from Perusio’s repository rather than directly from the source is we get the hard work of conforming to the Debian layout for free.

The answer is that Nginx since version 0.8.44 does not cache backend responses if they have a “Set-Cookie” header. This makes perfect sense because you don’t want a response which sets a cookie to be cached for subsequent requests, but I was stupid because I had totally forgotten that my Django app was using a POST form for all responses for non-authenticated clients (due to how Django’s CSRF middleware does its stuff).

The solution was to change the app so that it uses the GET method on the form in question, which in this case is fine from a security point-of-view.

The moral of this story is I should pay attention to my HTTP response headers and that I am badly short-sighted both figuratively and literally. With that fixed the site has gone from 15 requests per second to ~2000 requests per second!

Why does the web site promoting Microsoft Office for Mac OS X use, by default, a font that no Mac OS X user has on his or her system?

Well, the branding for Mac Office uses Segoe so it makes sense to keep that brand consistent across all the pages. But it was more of a rhetorical question, wasn’t it?

But Pierre has missed that the text is not displayed using a font specified by the stylesheet (he notes that he doesn’t have Segoe installed). What actually happens is it is rendered as canvas elements, one for each word.

This works using a JavaScript library called Cufón which aims to allow text to be rendered using any font the designer specifies, even if the font is not installed on the visitor’s computer. In the good old days the text would have been rendered as a graphic (as the designer did for the ‘Office:mac’ logo at the top of the page) but modern Web designers are too cool for that.

The advantage of the crazy download-able fonts and client-side JavaScript approach is the headline text remains accessible, you can copy and paste it etc. And at larger sizes it looks pretty good; only the smaller headline text renders poorly.

This screenshot shows how well it works for large size text and poorly for the small text (taken in Google Chrome dev version). I’ve selected part of the text to demonstrate that it is “live”:

Pierre is right, the smaller headline text does look terrible for the very same visitors Microsoft wants to sell to. But I don’t mind too much because the whole site is much less of a useability nightmare since the re-design. Microsoft’s Mac Web designers aren’t entirely evil.

Why did Apple allow the new App Store application to ignore the human interface guidelines? It isn’t like the toolbar is doing anything radically different to other information browsers, nothing that might warrant exploring new interface ideas.

It just looks odd, an arbitrary inconsistency that would be simple to correct.

Most of this was culled from a thread on the minidlna forum.

First install each of the following ports. The command for each would be something like sudo port install libiconv.

• libiconv
• sqlite3
• jpeg
• libexif
• libid3tag
• libogg
• libvorbis
• flac
• ffmpeg

Then check out the Mac branch of the current minidlna source from the CVS repository.

cvs -d:pserver:anonymous@minidlna.cvs.sourceforge.net:/cvsroot/minidlna checkout -r osx_port minidlna
cd minidlna

The current build script appears to miss out pulling in libiconv so I had to edit configure.ac, inserting a line to bring in libiconv.

AC_CHECK_LIB([iconv], [main],, AC_MSG_ERROR(Cannot find required library iconv.))

Now the build will work. Although I found I needed to run autogen.sh twice for it to generate all the necessary files.

source ENVIRONMENT.macports
sh genconfig.sh
sh autogen.sh
sh autogen.sh
./configure
make

This spits out the minidlna executable and a basic configuration file. Copy these to wherever you want them. Edit the minidlna.conf file, pointing it at the files you want to serve. There are examples of what to do in that configuration file.

And for testing purposes you can start the server from the build directory.

./minidlna -d -f minidlna.conf

Bingo.

I did try using ushare, another DLNA server, but I couldn’t figure out how to persuade my Sony telly to successfully connect to it. So I gave up. I feel it is useful to give up quickly when something doesn’t work until you run out of alternatives, I consider this triage. I also consider my telly’s inability to work with ushare and the fact that the telly will only play a very limited set of video formats a mark against the promise of DLNA.

(If you were mad keen on Django’s shift to class-based views you might reckon life wouldn’t be complete with a Bottle-based application until you employ classes for views. However Bottle’s use of decorators for tying URLs to views means it is less a natural fit than the same thing in Django.)

The problem is that you can’t just decorate the method in your class using bottle.route because if you use that decorator on a method in a class you are telling Bottle to use the method before it has been bound to an instance of that class.

So although I wish it did, the following example will not work:

import bottle

class ViewClass(object):
    @bottle.route("/")
    def home_view(self):
        return "My home page."

obj = ViewClass()
bottle.run()

Running that will lead to errors about not enough arguments passed to the view method of your ViewClass instance.

Instead you need to register the route right after the object is created. This can be done in the class’s __new__ method:

import bottle

class ViewClass(object):
    def __new__(cls, *args, **kwargs):
        obj = super(ViewClass, cls).__new__(cls, *args, **kwargs)
        bottle.route("/")(obj.home_view)
        return obj

    def home_view(self):
        return "My home page."

obj = ViewClass()
bottle.run()

It works. It isn’t that pretty. You could achieve exactly the same thing by explicitly passing the obj.home_view method to bottle.route after the instance is created. The advantage to doing this in the __new__ method is it will happen automatically whenever ViewClass is instantiated.

And if you go down this path then you should be aware of threads. Hey! Nice threads! Also I have a cold.

Looks like it is the work of Rusty Myers and Nate Walck (so far).

However it is another 15 seconds from login to a usable desktop, running Mac OS X 10.5.8. My Macintosh Classic with an 8 MHz 68000 processor is slightly faster booting System 6.0.8. Both the mini and the classic can run some version of Eudora for e-mail. Neither machine will run Google’s Chrome browser so I reckon it is even splits on which is the more useful.

Also I don’t need an adaptor to plug in the world’s greatest aircraft carrier masquerading as a keyboard on the classic. Little baby Jesus to think that Apple used to sell their considerably expensive computers with no keyboard, no screen, no graphics card, no memory, no hard disk, just a mouse! Progress means you can buy a new MacMini and it don’t come with a mouse neither.