Matthew Garrett

Why SHMConfig is off by default

Fri, 20 Nov 2009 16:56:40 GMT

Bastien mentioned the Chromium OS xorg.conf file, which includes an irritating wart - namely, Option "SHMConfig" "on". This tells the Synaptics touchpad driver to export its configuration data to a shared memory region which is accessible to any user on the system. The reason for this is that in the past, there was no good way for configuration information to be passed to input drivers through the X server at runtime. This got fixed with the advent of X input properties, and synaptics can now be configured sensibly over the X protocol.

But why was it off by default? Because, as I said, the configuration data is exported to a shared memory region which is accessible to any user on the system. And while it contains a bunch of information that's not terribly interesting (an attacker being able to disable my touchpad or turn on two finger emulation may be a DoS of sorts, but...), it also contains some values that are used to scale the input coordinates. Which means that anyone with access to the SHM region can effectively take control of your mouse. The current position is exported too, so they can also track all of your mouse input.

Now, this isn't stunningly bad. The attacker can only do this while you're touching the pad. You'll see everything that happens as a result. There's no way to fake keyboard input. They need to be running code as another user on the system - if they're running as the logged in user then they can already do all of this. And for a device as single-user as Google seem to be looking at, it's obviously not a concern at all.

But there's still plenty of places on the web suggesting that you enable SHMConfig, and various distributions that ship with it turned on (Ubuntu on the Dell mini used to, but got turned off after I contacted them about it). It's absolutely fine to do this as long as you're aware of the security implications of it, but otherwise please use X input properties instead.

Sigh.

Thu, 19 Nov 2009 20:32:10 GMT

If only eeepc-laptop sent standard keycodes, or something.

Oh, wait.

Writing a Linux distribution is hard. There's a huge range of interconnected dependencies. It takes a long time to learn how everything fits together, and fixing things properly rather than adding device-specific hacks often requires rewriting a lot of code. I'm sure Google will figure it out in time[1], and I'm also sure that the majority of their work is going into their UI rather than the underlying infrastructure. But even so, don't expect that you'll be able install Chromium OS on a random piece of hardware and have it work as well as, say, Fedora in the near future.

[1] Based on that script, I'd say they're about equal to Xandros at the moment

Legacy PC design misery

Fri, 13 Nov 2009 14:32:35 GMT

I've spent chunks of the last couple of days fighting a problem that's existed for about 25 years. The 8086 was a 16-bit processor with a 20-bit address space, limiting the maximum physical address that could be accessed to 1MB. However, quirks of the segmented memory system meant that addresses greater than 1MB could be constructed - these would wrap around to the bottom of memory. Because loading the segment registers was a time consuming operation, some programmers used this behaviour as a performance optimisation.

The 80286 introduced 24 bit address space. Unfortunately, this meant that the addresses that previously wrapped to the bottom of memory now pointed at real addresses - not ideal if you were expecting the old behaviour. IBM fixed this by tying the 21st address line (A20 - they're zero indexed) through an and gate, with the default behaviour being to keep it tied at 0 and thus maintaining the old wraparound behaviour. Applications that wanted to access the full address space needed to enable the A20 logic gate. IBM didn't want to add any extra hardware to their system if they could avoid it, so tied the other side of the and gate to a spare pin on the keyboard controller. By writing a couple of bytes to the keyboard controller, your PC-AT stopped pretending to be an XT and gave you access to all of the insanely expensive RAM it had stuffed in it. Hurray!

PCs have been emulating this behaviour since the AT was first cloned. Of course, this being the PC industry, many have got it wrong. There's a set of approaches for controlling the A20 gate that may work, varying in terms of performance and desirability. Most hardware will give the desired result (ie, I have no desire to run DOS executables from 1982, make my A20 work damnit) using any of the various methods of A20 enabling. Some hardware doesn't. The most common method used in bootloaders (where we still have access to system BIOS services) is to call int 15h with an ax of 0x2401, which asks the BIOS to enable A20 for us. This isn't implemented on all hardware, but we should get a failure back that lets us go and bang on the keyboard controller in an attempt to get it to pay attention[1].

Enter the Kohjinsha SC3.

I picked this up second hand in Japan. It's a ridiculously cute little tablet, only slightly larger than hardware that's comfortably in the MID range. It booted a Fedora liveCD perfectly, though having GMA500 graphics meant that what appeared wasn't terribly attractive. Installation proceeded happily enough, followed by a reboot and... nothing. Grub loaded the kernel and initrd, jumped to the kernel and everything hung.

So, for the past couple of days, I was stepping through the kernel setup code, trying to work out where and why it was hanging. I'd got it narrowed down to the region where the kernel tried to free the memory used by the initramfs, but the failure hopped around depending on my kernel build. Something was clearly very wrong. The strangest thing about this was that if I booted the liveCD boot menu and selected "Boot from local drive", everything worked perfectly. isolinux was clearly doing something that grub wasn't, but there's rather a lot of code to step through there.

Things became a lot easier once I found that the OpenSuse version of grub worked. Their grub has a rather smaller set of patches than ours, and only a few looked even plausibly relevant. It only took ten minutes or so to figure out that it was one that altered the A20 code. Things became much clearer then.

The main functional difference between the Suse A20 implementation and the upstream one[2] is that the Suse one explicitly tests whether the A20 enabling worked by putting values at two different addresses that would be the same if A20 is disabled. By comparing them, we know whether A20 is working properly or not. If not it can then fall back to other mechanisms. The Fedora code trusted the BIOS's claim that the int 15 call had worked. The Kohjinsha's BIOS lied, A20 remained disabled, grub copied the kernel and initramfs to chunks of address space that contained lies rather than RAM and everything fell over horribly.

Thankfully, not a difficult fix once the problem was identified. But seriously, people. How hard can it be not to screw this up?

(For an excrutiatingly detailed analysis of how hard it can be not to screw this up, see here)

[1] the Intel Macs don't implement the int 15 approach, but return a failure. They also don't have a legacy keyboard controller, so attempting to hit that resulted in grub falling over. The magic IO port approach works. Another example of how the Intel Macs aren't really PCs...

[2] grub2 implements the more paranoid check

The ACPI Embedded Controller

Tue, 10 Nov 2009 14:58:52 GMT

Of course, the event model I described before is far too simple to be worthy of a place in the ACPI spec. At the most basic level, there's more possible events than there are GPEs to attach them to, so there's a need for some further complexity. This manifests itself in the form of the ACPI embedded controller (EC).

The EC is typically a small microprocessor sitting on your motherboard, often implemented in the same hardware as the keyboard controller. It shares a lot in common with the keyboard controller - on PCs it'll usually appear in system io space, with one register for writing a command or reading a status, and a second register for passing data back and forth[1]. There's 256 registers available, so a typical interaction might be to write the READ command (0x80) to the command register, write the EC register address to the data register and then read back from the data register to get the EC register contents.

The embedded controller will often be responsible for tracking information about the hardware, such as the temperature. Attempting to read the temperature through ACPI will execute an ACPI method - in the case of the temperature being monitored by the embedded controller, this method will attempt to read from an EC register. The EC driver then performs the read and returns the result, which gets converted into decidegrees kelvin and passed back to whatever made the temperature query.

But, as mentioned above, the EC also generates events. These may be in response to a user initiated event like a hotkey press, or may be triggered by some change in hardware state like a thermal trip point being passed. The embedded controller will then raise a GPE.

Unlike normal GPEs, the EC GPE is not handled by looking for a _Lxx or _Exx method. Instead, the ACPI tables provide information about the GPE that the EC is using. This may be in the form of a _GPE definition in the EC object in the main ACPI tables, or alternatively may be provided in an ECDT (Embedded Controller Descriptor Table), an optional table that provides all the EC information. In either case, the OS knows which GPE will be triggered by the EC. It then installs a handler that will be called whenever the EC raises that GPE.

Things get a touch confusing at this point. The first thing this handler does is read the command byte, which functions as a status byte on reads. It then checks whether the SCI_EVT bit is set. This informs the system that the GPE was in response to a hardware event, and so the EC handler writes a query command to the EC command register and then reads back a value between 0 and 255 from the data register. This is then mapped to a _Qxx method, with xx representing the number of the EC event read from the data register. Like the _Lxx and _Exx methods, the _Qxx method is then executed.

The problem with all of this is that the EC isn't that fast. When a byte is written to it, it's necessary to read back the status byte and check whether the IBF bit is set. This is set when the OS writes a byte to the data register, and cleared once the EC has processed it. The straightforward way to deal with this is to poll the status byte until the bit is cleared, and then write the next byte, but polling is slow and wastes CPU time. The EC can instead be set to interrupt mode, where it'll fire a GPE when the IBF bit clears.

The EC has one additional function. The ACPI spec allows for an i2c bus to be implemented through the EC, with EC registers mapping to i2c registers. The observant among you will realise that this means that there's an indexed access protocol being implemented on top of indexed access hardware, which is more layers of indirection than seem sane. For additional humour, this is usually only used to add support for ACPI smart batteries. ACPI batteries are generally abstracted behind a set of ACPI methods that provide information. Smart batteries instead speak i2c directly to the OS[2] for no real benefit. Linux handles these devices fine, and while the chances are you probably don't have one, the chances are also that if you do you haven't noticed.

The final quirk of ACPI events is that there's yet another means of delivering events. The term "fixed feature" is used to describe an ACPI device that isn't described in the ACPI tables. A power button may be implemented as a fixed feature device rather than a normal ("control method") device. This is indicated by a flag in the fixed feature block. Hitting a fixed feature power button will generate an ACPI interrupt, but no GPE. Instead the OS has to read the fixed feature block and note that the power button flag is set there. It then notifies userspace appropriately. Sleep buttons can also be implemented this way, but other devices will be in the normal ACPI tables and will generate either GPEs or EC events.

[1] On my laptop, these are ports 0x62 and 0x66 - compare to the keyboard controller's use of ports 0x60 and 0x64

[2] As directly as indirection via the EC can be...

ACPI general purpose events

Tue, 10 Nov 2009 03:08:36 GMT

ACPI is a confusing place. It's often thought of as a suspend/resume thing, though if you're unlucky you've learned that it's also involved in boot-time configuration because it's screwed up your interrupts again. But ACPI's also heavily involved in the runtime management of the system, and it's necessary for there to be a mechanism for the hardware to alert the OS of events.

ACPI handles this case by providing a set of general purpose events (GPEs). The implementation of these is fairly straightforward - an ACPI table points at a defined system resource (typically an area of system io space, though in principle it could be something like mmio instead), and when the hardware fires an ACPI interrupt the kernel looks at this region to see which GPEs are flagged. Then things get more interesting.

The majority of GPEs are implemented in the ACPI tables via methods with names like _Lxx or _Exx. The xx is the number of the GPE in hex, while the leading _L or _E indicates whether the GPE is level- or edge-triggered. If an ACPI interrupt is fired and GPE 0x1D is flagged as being the source of the interrupt, the ACPI interpreter will then look for an _L1D or _E1D method. Upon finding one, it'll execute it. What this method does is entirely up to the firmware - on most HP laptops, GPE 0x1D is hooked up to the lid switch[1] and so executing it will send a notification to the OS that the lid switch has changed state. The OS will then evaluate the state of the lid switch (generally by making another ACPI query) and send the event up to userspace.

How does the lid end up triggering GPE 0x1D? Things get pretty hardware specific at this point. Intel motherboard chipsets have a set of general purpose io (GPIO) lines that can, for the most part[2], be used by the system vendor for anything they want. For a lid switch, one of these lines is hooked to the switch and the BIOS configures the GPIO as an input. Pressing the switch will cause the GPIO line to become active. The GPIO lines are mapped to GPEs in a 1:1 manner, though with an offset of 16 - ie, GPIO 0xd will map to GPE 0x1d. If GPIO 0xd becomes active, GPE 0x1d will be flagged and an ACPI interrupt sent. The ACPI code will then do something to quash the interrupts, such as inverting the polarity of the GPIO[3], as well as send the notification to the OS.

Why are the GPIOs offset by 16 relative to the GPEs? The lower 16 GPEs (again, talking about Intel hardware) have pre-defined purposes[4]. These range from things like "Critically low battery" to "PCIe hotplug event" down to "This device triggered a wakeup". And the latter is what I'm most interested in here.

Various pieces of modern hardware can be placed into power saving states when not in use. The problem with this is that the user experience of having to turn on hardware before you can use it is not a good one, so in order to make this the default behaviour we need the hardware to tell us that something happened that requires us to wake the hardware up.

There's something of a chicken and egg problem here, but thankfully most of the relevant modern hardware has out of band mechanisms to tell us about things going on. The PCI spec defines something called Power Management Events (PME), which are driven by an additional current that's supplied to the hardware even when it's otherwise turned off. On plug-in PCI Express cards, firing a PME generates an interrupt on the root bridge and a native driver can interpret that, but for legacy PCI devices and integrated chipset devices the notification has to come via ACPI.

The example I've been working on is USB. It's a good choice for various reasons - firstly, there's already support for detecting when the USB controller is idle. Secondly, modern USB host controllers have support for generating PMEs on device insertion, removal or (and this is important) remote wakeup. In other words, as long as the USB bus is idle we can power down the entire USB controller. If the OS tries to access a USB device, we'll power it back up. If the user unplugs or plugs a device, we'll power it back up. If a previously idle device suddenly responds to some external input, we'll power it back up. And it's all nicely invisible to the user.

How does this work? The controller retains a small amount of power even when nominally pwoered down. This is used to keep the detection circuitry alive. When it receives a wakeup event, it asserts the PME line. The chipset detects this and fires a GPE. The OS runs this GPE and receives a device notification on the ACPI representation of the USB controller, telling us to power it back up. We do so and process whatever woke us - if the bus then goes idle again, we can power down once more.

The astonishing thing is that this all works. The only problem we have is that it relies on the machine vendor to have provided the ACPI methods that are associated with the GPEs. If they haven't, we can't enable this functionality - even though the hardware is capable of generating the GPEs, we have no method to execute to let us know which device has to be woken up. The GPE is never answered, we never acknowledge the PME and the hardware keeps on screaming for attention without getting any. And, more to the point, it never gets powered up and your mouse doesn't work.

There's a pretty gross hack to deal with this. In general, we know what the GPE to device mappings are - they're pretty static across Intel chipsets, and while AMD ones can be programmed differently by the BIOS we can read that information back and set up a mapping ourselves. This trick also comes in handy when some vendors (like, say, Dell) manage to implement one of the GPE events wrongly. Everything looks like it should work, but the method never sends a notification because it's buggy. In that case we can unregister the existing method and implement our own instead.

This code isn't upstream yet, but patches have been posted to the linux-acpi mailing list and with luck it'll be there in the 2.6.33 timeframe. My tests suggest about 0.2W saving per machine, which isn't going to save all that many polar bears but seems worth it anyway.

[1] _L1D = lid. Sigh.

[2] There's a few that are reserved for specific purposes

[3] So where before it had to be high to be active, it now has to be low to be active - this means that it'll now trigger on the switch being opened rather than closed, so you'll get another event when you open the lid again.

[4] You can find a list in the documentation for the appropriate ICH chip - the relevant section is "GPE0_STS" under the LPC interface chapter.

Looking to the past

Mon, 09 Nov 2009 20:56:21 GMT

It’s an oft-voiced suggestion that rather than looking at the bad things that happen in our communities, we should focus on the good things. There’s a number of highly successful geek women already – should we not be concentrating on encouraging more of them, rather than scaring people away with tales of thoughtlessness, discrimination and outright abuse?

Let’s draw an analogy. One day, a $20 charge appears on your credit card. You didn’t make it. You report it to your credit card company, who assure you that they take fraud seriously and then do nothing. A few days later, another $20 charge. Your credit card company tells you that such events are rare, unrepresentative of the general credit card experience and continue to do nothing. A week afterwards, another charge. This time your credit card company describes how they’re planning on implementing a brand new anti-fraud system, but that this is unrelated to any events that may currently be occuring and will give no details as to when it’s going to be rolled out. And proceed to ignore any further reports you make about fraudulant transactions.

Would you stay with this company? Or would you take your business somewhere else?

The problem with the “Let’s look to the future rather than spending too much time getting stuck in the present” argument is that it assures people that things will get better without providing a roadmap for getting there. It does nothing to validate their concerns or make them feel wanted within a community. It assumes either that people will stick with a community that doesn’t respond to their complaints, or that it’s possible to construct a community that’s welcome to an assortment of genders, ethnicities and lifestyles without any of those people being represented in the first place.

Ignoring people’s concerns is an excellent way to drive them away from your community. Doing so because of a potential future that’s probably conditional on you having those people in your community is short sighted and self defeating. Ignoring the present doesn’t benefit the future. It benefits the status quo.

(Originally posted here)

More GMA500

Wed, 28 Oct 2009 18:05:16 GMT

But is Intel really the party at fault, here?

For shipping a gpu without open drivers? Given that the alternatives involve someone else designing, fabbing and releasing a piece of hardware under Intel's name without being sued in the process, I'm going to have to say "Yes".

(Note that while Moblinzone.com is a website owned by Intel, the writers don't appear to be Intel employees)

Asymmetries in offence

Mon, 12 Oct 2009 18:30:07 GMT

I wasn't going to write about this since I thought that Chris's post covered pretty much everything I would have said, but after reading Scott's entry on how people would have interpreted Mark's remarks differently if he'd said "We'll have less trouble explaining to boys what we actually do" instead I realised that people are still confused about the fundamental issue here.

The assumption that Scott's making is that "girls" and "boys" are semantically equivalent in this case. They're not. There's various ways in which the symmetry is broken, but the most basic one is that Mark's a straight man. When the overwhelming stereotype is that "we" as a community are heterosexual males, using "we" as a shorthand for "People who are straight men" is unfortunate because it supports that stereotype. Using "we" as a shorthand for "People who are attracted to men" doesn't. Unsurprisingly, this results in a fairly significant change in who's going to be offended.

Whatever his intentions (and I could easily believe that it was a slip of the tongue), Mark managed to imply that the Linux community is entirely made up of straight men. This is possible because straight men do make up the majority of the Linux community. In contrast, Scott's version doesn't succeed in implying that everyone in the Linux community is attracted to men because it's blatantly obviously not the case, so we know that Scott is using "we" in a different manner. Context is important, and unless you can invert everything else about the situation as well then simply replacing the word "girls" with "boys" doesn't give you any meaningful insight into whether or not people are justifiably offended.

In a more general sense, I'm saddened by this case because I think it's a clear case where the Ubuntu code of conduct could have been used to good effect. "Be excellent to each other"[1] ought to include accepting that you've offended other people without meaning to and making appropriate restitution. If the offence was unintended, an apology should be cheap. Whatever the reality of the situation, failing to provide that apology gives people the impression that either the offence was intended or that Mark doesn't care about those who were offended. That's not a good way to build an inclusive community.

[1] Mako's original summary of the code of conduct

Intel IGD opregion and GMA500

Thu, 24 Sep 2009 18:24:23 GMT

A while back, Intel defined a specification for binding ACPI-defined methods for controlling hardware to the OS-specific driver, ensuring that the two don't get out of synchronisation. I added support for this to the in-kernel i915 driver last year, and after a couple of awkwardnesses it works well now. One consequence of this that showed up slightly later is that it's necessary to do some of the setup from the i915 driver rather than the ACPI driver, which meant that we had to defer the ACPI driver from binding until the drm driver had done that setup.

The problem with GMA500 is that it also implements the IGD Opregion spec, and the ACPI video driver detects this and refuses to bind. But the GMA500 kernel driver doesn't implement support for the spec and so doesn't call the function that triggers the ACPI video registration. Working around this is simple - just add acpi_video_register() to the init function of the GMA500 drm. But note that this means that you're failing to implement the spec properly, and there's potential for stuff to be broken. A full implementation of the spec for GMA500 wouldn't be especially difficult, but there's no docs and I have no hardware so I'm not going to do it myself.

The reason I bring this up is that various people have been approaching this problem in a different way. It's easy to assume that the check in the acpi driver was naively assuming that all Intel hardware was driven by i915 and that this patch was broken. It's actually entirely correct and the (out of tree) GMA500 driver was broken. If Intel had made the effort to get their code properly upstream, it'd have been fixed there when the original change was made and nobody would ever have had a problem. Just say no to out of tree drivers.

Portland

Thu, 17 Sep 2009 00:04:36 GMT

I'm off to Boston in under 16 hours, and I'll be getting into Portland around lunchtime on Monday. I'll be talking at Linuxcon about how we're broadening power management on Linux to be applicable from phones through netbooks up to supercomputers - that's 10:15 on Tuesday. At 10AM on Friday I'll be presenting at the Linux Plumbers conference on how userspace can express its requirements to the kernel more clearly, thereby allowing the kernel to be smarter about powering down hardware. And after a short hop down to SF for the weekend, I'll be back in Portland at the X developers conference talking about the role of X in providing relevant information to the kernel and using that to facilitate more aggressive power management.

Three talks in under 10 days. I'll even do my best to ensure that there's new jokes for each of them.

Bye

Tue, 15 Sep 2009 11:16:36 GMT

I'm moving to the US on Thursday, so I will be here on Wednesday evening from about 7. If your presence is unlikely to make me stupifyingly angry, feel free to join me.

Mon, 07 Sep 2009 14:13:20 GMT

Good thing: Vicodin
Bad thing: The broken arm that necessitates the vicodin

Tue, 25 Aug 2009 11:31:12 GMT

I moved out today, and in a bit over a month should be firmly relocated in Boston. Which means a bit over a month of living and working out of bags, but I'm in a park, it's sunny and I'm relaxed enough that even working through Bugzilla isn't making me sad.

Defective by Design

Tue, 11 Aug 2009 20:18:04 GMT

You know what's Defective by Design? Thinking that this kind of functionality is a good thing, resulting in this.

(Update: The "Forward this page to someone" functionality's been disabled on the site, which solves that problem nicely. Thanks to the admins for taking care of this)

Mon, 27 Jul 2009 18:32:04 GMT

Though people might be forgiven for having gained the opposite impression, I don't yearn for a future where open source development involves spending 25% of the time coding and 75% of the time where we have earnest conversations about the precise social nature of our utopian society and hold votes on whether to send polite but firm telegrams to people who accidently swore at some point in the recent past. I've never really yearned to live in a commune. I hate the idea of singing around campfires and there are many people who I never, ever want to be polite to.

No. What I want is a future where I can say this person is being a dick, or this person is being a dick or that this person is being a dick and not have to engage in lengthy explanations as to why before people agree. A future where calling someone on their behaviour makes people examine that person's motives rather than yours. A future where you're allowed to criticise without being perfect. A future where people don't feel scared to speak up because the consequences of doing so may include people posting their address and phone number and trying to get them fired. A future where disagreements about behaviour aren't so often intrinsically linked to gender. A future where I get to stop writing about any of this because it's taken for granted anyway.

And I want some awesomely good arguments for why this isn't an excellent future that we should all be encouraging.

Intel graphics in rawhide

Mon, 27 Jul 2009 03:09:55 GMT

I've just committed some new Intel driver code to rawhide, which should be in the next kernel build. There's a chance that some people with Intel laptops will see some screen flickering. If you do, could you please file a bug against the kernel in the Red Hat bugzilla and make sure you Cc: me (mjg@redhat.com). Include the output of the xrandr command.

LCA 2010

Fri, 24 Jul 2009 02:47:16 GMT

Depending on timezones, you currently have somewhere between 8[1] and 33[2] hours to submit a presentation for this year's Linuxconf Australia[3]. If you think you have something interesting to talk about, then do it and enjoy an unarguably top-tier Linux conference. Also, New Zealand is excellent.

[1] Kiribati is weird.
[2] For all those of you stuck on Howland or Baker islands. If you start swimming now you might reach civilisation before the conference starts.
[3] In New Zealand, obviously

RMS and virgins

Thu, 16 Jul 2009 00:44:30 GMT

Many of the comments here and here are disheartening, but part of the problem is that many people didn't see the presentation. Dave linked to a previous iteration of the same presentation[1]. Here's a transcription (errors are mine and mine alone):

I am Saint Ignucius of the church of emacs. I bless your computer, my child. Emacs started out as a text editor. An extensible text editor, which became a way of life for many users because it was extended so much they could do all their computing work without ever exiting from emacs. And then, it became a religion, with the launch of the newsgroup alt.religion.emacs. Tday in the church of emacs we have a great schism between several rival versions of emacs, and we also have saints. But fortunately no god, instead of gods we worship an editor. To be a member of the chuch of emacs, you must recite the confession of the faith. You must say there is no system but gnu, and linux is one of its kernels.

Then if you become a hacker you can celebrate that by having a foobar mitzvah, a ceremony in which the new hacker stands in front of the assembled congregation of hackers and chants through the lines of the system source code. And we also have the cult of the virgin of emacs. The virgin of emacs is any female who has not yet learned how to use emacs. And in the church of emacs we believe that taking her emacs virginity away is a blessed act.

The church of emacs has certain advantages compared with with other churches I won't mention. For instance, to be a saint in the church of emacs does not require celibacy. Although for some of us hackers we wouldn't notice the difference. But it does require living a life of moral purity. You must exorcise whatever proprietary evil operating systems have posessed the computers under your control or set up for your use and then install a wholy free operating system and then only install free software on the system. If you make that vow and you live by it then you too will be a saint, and you too will have the right to wear a halo. If you can find one, because they don't make them any more.

Sometimes people ask me whether it a sin in the church of emacs to use the other editor, vi. It's true that vi vi vi is the editor of the beast. But using a free implementation of vi is not a sin, it's a penance. And sometimes people will ask whether my halo is really an old computer disk. This is no computer disk, this is my halo! But it was a computer disk in a previous life. So thank you very much.

One of the frequent counterarguments against this being sexist is that RMS has often spoken out against sexism (see here, for example). It's very easy to claim to be free of sexism. It's much harder to perform the degree of introspection required to understand whether any of your actions are motivated by viewing genders differently. Do I believe that Richard is attempting to deliberately denigrate women? Not in the slightest. But I also don't believe that someone entirely gender-blind would have made the above joke.

My point here isn't to claim that he's a bad person as a result. I've got personality flaws large enough that you could probably drive a bus through them, but I'd be slightly upset if people thought I was evil because of them. My point is that nobody is above criticism, and if someone behaves in a way that offends a large subset of the community then they should to be criticised. Failing to do so sends the signal that we don't care about those who were offended, and at the same time provides no incentive for people to change their behaviour as a result. And yes, I think those who have high profile positions in the community should be held to higher standards than others - Richard's comments on Mono carry more weight because of who he is, but the cost of this is that everything else he says does as well. And if one of our nominal leaders is perceived as sexist then that reflects badly on all of us.

[1] Note that his GCDS presentation did not entirely consist of this routine - there was also significant discussion of Mono and why he believes that adopting it is dangerous. I don't entirely disagree with him, but that's really not the point here. I'm not involved in any Mono development. I work for a company that ships Mono in the community distribution it supports, but not in the enterprise distribution that pays my wages. If anyone brings up Mono in any comments here they'll be blocked and the thread deleted, because it is not relevant to this discussion.

Simple conference organisation suggestion

Sat, 11 Jul 2009 05:32:17 GMT

If you find the comments discussed here unacceptable, don't invite RMS to keynote at your conference without an explicit apology and expression of understanding from him beforehand. I'm seriously at the end of my patience with people being unwilling to call others on behaviour they perceive as unacceptable. Either make it obvious or accept that people will treat your failure to do so as implicit support.

Edited to clarify that I think the apology should be from RMS, not from people who invite him to keynote

Chrome OS

Fri, 10 Jul 2009 15:47:11 GMT

It turns out that I was entertainingly wrong a while ago, though I persist in claiming that this is an utterly ridiculous idea and I should be forgiven for thinking that Google were sane. The whole thing really still doesn't make sense to me. Worthwhile support of hardware is difficult. I'm going to take it as a given that Google aren't going to claim to support arbitrary hardware. People who would never otherwise try Linux will install it, and the state of many Linux drivers is sufficiently poor that it'd do a great deal to damage their brand. The logical assumption is that it'll be available pre-installed on devices where Google have worked closely with the hardware vendors.

Which concerns me somewhat. History isn't filled with compelling examples of this. Xandros's low-level support for the Eee mostly seemed to consist of a pile of shell scripts made of cheese and failure. The bizarro-Linux on the hilariously dodgy MIPS-based netbook I have is about as functional as my wisdom teeth. The best example of a Linux vendor working with OEMs is probably Canonical, and their enthusiasm for merging hardware support code in their OEM-specific distributions has led to things like touchpad gesture support based on using a known security hole or drivers that reimplement one that's already mainline.

What I'm trying to say here is that pretty much every desktop Linux product based on cooperation between OEMs and an existing Linux vendor has been built on top of a tower of shit. That's partly because it's a hard problem, but it's also because most OEMs produce dreadful Linux code and the Linux vendors don't have the resources to rewrite it in a clean way in the timescale permitted between hardware being finalised and shipping product. I haven't seen Google recruiting a larger than normal number of people with Linux distribution experience lately, so I suspect that the situation may be the same there. This is probably fine if the number of products is relatively small - there's an opportunity to QA them sufficiently to ensure that the rough edges underneath don't accidentally take someone's hand off. But otherwise there is a genuine risk that poor-quality devices will appear with Chrome OS and people will blame Google for the poor user experience.

So it seems like a risk for Google. Either there'll be a small number of devices and the same vague level of discontent that surrounds the fact that the number of shipping Android devices doesn't seem to have reached expectations, or a large number of potentially crappy devices. What's the payoff? It's pretty clear that this is going to be based heavily around Google's web apps, possibly with disconnected operation. That gets Google a lot of lockin. It also neatly sidesteps the entire disaster that has been providing application add-ons and updates for netbook Linux distributions. But it seems that Google could have achieved that by partnering with a distribution that already has experience in this field rather than going it alone.

I remain unconvinced that this is a sensible decision for Google. But then, I've already demonstrated that I don't have the faintest idea what's going on. Which probably puts me in the same field as most of the analysts I was bitching about before. Zounds. The irony.

What does the desktop want from the kernel?

Mon, 06 Jul 2009 18:07:00 GMT

I'll be running a session on Wednesday at GCDS to find out what desktop developers would like to see from the kernel. There's a lot of interest in making things easier for file indexers, but if anyone has other problems that could be made easier with some level of kernel support then please turn up. No precise time or location yet, but probably around 3PM at the university. More details forthcoming.

Mon, 06 Jul 2009 17:18:10 GMT

One of the strengths of the open source community is that so much happens in the open. It's generally easy to find out what's happening in a project and directly interact with the developers. Code is out in the public. People frown upon closed discussion and implementation. But there's also a cost. Personality conflicts get hidden in the corporate world. We air them in public. And while in some ways that's arguably an advantage, it also results in things like this.

Now, to be fair, I've never been an especially big fan of Sam's work. His journalism generally leans towards lazy sensationalism rather than any attempt to actually understand the issues. He's not especially well versed on the basis of free software (see his assertion that the difference between the cathedral and bazaar development models is about project leadership rather than source code availablity and how people participate). He's managed to mischaracterise my opinions in the past, which means trusting his characterisation of anyone else is somewhat difficult. But that's not the point. Sam's article isn't about facts or analysis. It's about crucifying someone for expressing an opinion that Sam disagreed with.

I don't know Anirudh. From his website, he sounds like a pretty typical young hacker. He's contributing to a project that interests him. He's self taught. He holds strong opinions. And, last week, he published a rant on a topic he cared about - specifically his feeling that the patent concerns about Mono shouldn't prevent people from developing in it, and that RMS's statements about not using Mono harm the free software community more than they help it. I don't agree with all of his arguments. I don't think it was a hugely well structured rant. And, shockingly, telling RMS to fuck off isn't going to achieve a great deal.

But the point wasn't to change the world. People like voicing their opinions. I've done so several times on several occasions on a wide range of topics. People have disagreed with me. People have voiced concerns about the way I've expressed myself. People have flat out told me to stop being a cock. But nobody has attempted to tell me that I haven't earned the right to express that opinion. Nobody has expended four pages to tear me apart for daring to criticise another member of the community. Nobody has dared to call me a coward for deciding that I'd gone too far and modifying or retracting something I've written.

Yet that is precisely what Sam Varghese has done here. And let's be clear here - the use of the word "fuck" is a red herring. Sam is publically humiliating someone because he has his own agenda. He's picking on someone smaller than him because he can. He's explicitly stating that anyone arguing in favour of Mono can expect to be thoroughly abused in front of a large audience. In Sam's world I don't get to criticise the shockingly distasteful sexist remarks RMS made during his GCDS keynote because I didn't write emacs. I don't get to point out that ESR's understanding of racial genetics and natural selection are fundamentally flawed because I don't maintain the jargon file. I don't get to say that I think Ted Tso was wrong about ext4's semantic changes because I've never written a filesystem. And if I do any of these things, I can expect Sam to pop up and do his best to destroy my reputation.

Of course, he won't. Because in Sam's world, it's not actually about whether anyone has achieved great things. It's about whether the targets of his vitriol will be able to stand up for themselves or not. And while the truth is that nobody in the community is above criticism and nobody needs to earn their right to disagree, I doubt that Sam is ever going to show the courage that Anirudh did and publish a public retraction of his astonishing attack.

Shame on you, Sam Varghese.

Thu, 25 Jun 2009 21:38:42 GMT

I've had hard drives die before, but this one seems to be doing it in an especially pathological way. It started with the kernel throwing irq timeout errors and then stepped up into actual read errors, culminating in a corrupt journal, a read-only block device and a forced fsck. It's been behaving a little better since then, though occasional io stalls (without any kernel error) suggest that it's having to repeatedly retry some sectors. SMART says it's all fine, so obviously I'm backing it all up now before a new disk arrives tomorrow and I can sort out access to the data centre. Email might be a bit spotty for me until then.

Turns out that running a 2.5" PATA drive for approximately 4 straight years may not have been the best of ideas. Who'd have guessed?

Wed, 24 Jun 2009 14:07:17 GMT

I spent last week in the US, during which I discovered that my preconceptions of Albany as a post-industrial wasteland with no redeeming features were incorrect (it's actually a post-industrial wasteland with some decent bars), made minor contributions (mostly in the form of pesto) to a prize winning chalk picture and cycled on the wrong side of the road for the first time in a ridiculous number of years.

But more relevantly, I spent most of the week trapped in Westford. One of the things I've been looking into lately is USB autosuspend, a kernel feature which allows devices to be powered down when they're not in use. This is especially interesting for USB, since it's a poll-based protocol. If the end device is powered up then bus traffic will be generated, even if everything's idle. USB autosuspend allows this to be avoided and thus saves a worthwhile amount of power.

However, drivers need to support USB autosuspend before it's useful. Further, devices need to be able to cope with it. Some older kernel releases had autosuspend enabled by default, leading to all kinds of fun as people discovered that their scanners and printers had stopped working. This was fixed by leaving autosuspend disabled by default for most hardware. The first component that I'm working on is support for drivers to indicate whether or not a given piece of hardware supports autosuspend, allowing it to be enabled by default for that kernel. Handling this at a per-driver level means we shouldn't end up with obnoxious white or blacklists. The second is adding support for autosuspend to more drivers. Right now I'm concentrating on hardware that's common in laptops. There's support for autosuspend in the qcserial and uvc drivers, and some experimental (and unmerged) patches for bluetooth and some other modem drivers. Palm have just released their kernel code, which includes autosuspend support for the cdc-acm driver. I'm working on cleaning these up and getting them into a mergable state, and part of what I was doing in Westford was testing that various pieces of hardware work correctly. The good news is that they seem to, and with a bit of luck we'll try shipping support for a lot of this in F12. If that doesn't end up causing real world problems then it ought to be safe to merge it all to mainline.

The final component of this is handling autosuspend on devices with userspace drivers. Our only real choice there is for packages to ship udev fragments that enable it for working hardware. I've added support to the fprint package in rawhide, which means that fingerprint readers should now be powered down unless they're opened. Nobody seems to have complained yet, so I'm cautiously optimistic that this can be sent upstream without any problems.

Tue, 23 Jun 2009 20:41:44 GMT

It's now been over 6 months since Poulsbo hardware with Intel's GMA500 graphics core started shipping in volume. And we're still utterly lacking in any sort of worthwhile driver. It's an impressive turnaround from the recent days when the straightforward recommendation for mobile Linux hardware was "anything that has lots of Intel stuff in lspci", and while the Poulsbo situation in itself doesn't change that hugely it's potentially symptomatic of a worrying trend within parts of Intel.

The first thing to realise here is that, like most large companies, Intel consists of a large number of business units with different priorities. Their open-source technology center has historically had responsibility for providing Linux support for hardware, but this obviously depends on other business units cooperating with them. And there's strong evidence that many of those business units don't get it.

There's been signs of this for some time. Back before the days of the Intel X.org driver gaining native modesetting support, some people ran the Intel embedded graphics driver. This was (is?) a closed X driver that was able to provide native modesetting on platforms that could only otherwise be run at incorrect resolutions. One business unit was shipping a driver that was more functional than the official Intel Linux driver. To the best of my knowledge, none of that code was ever used in the rewritten Intel driver that now provides the same features.

Poulsbo is another example of this. Intel wanted a low-power mobile graphics chipset and chose to buy in a 3D core from an external vendor. IP issues prevent them from releasing any significant information about that 3D core, so the driver remains closed source. The implication is pretty clear - whichever section of Intel was responsible for the design of Poulsbo presumably had "Linux support" as a necessary feature, but didn't think "Open driver" was a required part of that. There's not a lot any other body inside Intel can do once IP-limiting contracts are signed and the hardware's shipping, but it ends up tarnishing the good reputation that other parts of Intel have built up anyway.

And while Poulsbo is the most obvious example of this to date, it's not the only one. Intel recently decided to make the EFI development kit discussion lists private. Various drivers for Moorestown (the followup platform to Poulsbo) have been submitted to the Linux kernel, and while they have the advantage of being GPLed they have the disadvantage of being barely above the level of typical vendor code. Objections that chunks of them simply don't integrate into Linux correctly has done little to get these problems fixed - I still have no real idea how the runtime interface to power management on the SD driver is supposed to be used, but I suspect the answer is probably "badly".

This all makes sense if you assume that there are large groups of people in Intel who don't talk to each other. But to the casual observer it just looks schizophrenic. Explaining to an irate user that the Intel who shipped a closed Linux graphics driver is only barely the same Intel who contribute so much to architectural improvements in the Linux graphics stack doesn't make their hardware work. And while all of this confusion is going on, Intel's competitors are catching up. Atheros are now making significant contributions to the state of Linux wireless. AMD are releasing graphics chipset documentation faster than Intel, and radeon support is improving rapidly.

Is the future going to be one where we can no longer simply say that Intel hardware will Just Work? Is their work on Moblin (easily the most compelling Linux UI for netbooks) going to be wasted on the broader Linux community because it'll mostly end up running on hardware that's not supported by the mainline Linux kernel? Does Intel have a real commitment to open source, or is that being lost in the face of short-term requirements?

Intel need to demonstrate that they have a company-wide understanding of what Linux support actually means or risk losing much of what they've earned over the past few years. I'm desperately hoping that Poulsbo and what we've seen so far of Moorestown are the exception, not the future norm.