a convenient truth

The government’s recent flirtation with setting a minimum price for alcohol has exposed some of the problems government’s have with drug policy generally.  Since the US declared a “Global War on Drugs” (GWoD) at the start of the last century, government’s throughout the world have struggled with finding a social policy on intoxicants that has successfully fulfilled some of their basic aims.

This hasn’t been helped by the lack of clarity about what their basic aims actually are.  The original intentions of the GWoD were twofold, both core forces in US politics: racism and religious puritanism.  Since the original social forces were so unpalatable, it’s not entirely surprising that their achievements have been so destructive.

Racism has always been a key component of laws on intoxicants.  It’s an obvious but sometimes overlooked fact that every cultural group has it’s own preferred intoxicants.  The preferred intoxicant amongst the vast majority of Europeans has always been alcohol, which is why it has generally been very lightly regulated.

However, if you belong to a less powerful cultural group, your access to your preferred intoxicants will have been severely limited since around 1900.  Cannabis, for example, was first made illegal in California.  This wasn’t because the drug itself is harmful (it basically is not), but because it was the preferred intoxicant of Mexicans, who at the time were moving to California in great numbers.  By targeting Cannabis, the authorities could legitimately abuse any Mexicans they saw.

Since then of course, intoxicants legislation has been variously targeted at blacks, hippies, flappers, ravers, hells angels and pretty much anyone else who is out of favour.

Gordon Brown was explicit about their aims in the climbdown on their alcohol price legislation. He said the aims of the legislation was in response to “the excesses of a small minority”, but that he didn’t want to punish the “sensible majority”.

In essence what he said is that it isn’t alcohol he has a problem with, it’s a specific group of people, and that they were attempting to discriminate against them specifically with a law on the use of alcohol.  Unfortunately for them, they are unable to target this group sufficiently, because of the use of alcohol in the general population.

You can be certain that if the people he didn’t like had chosen an intoxicant that wasn’t in general use amongst his own constituency, he would have happily promoted harsh penalties for it’s use, even if it was completely harmless.

Gary Pugh, director of forensic services at Scotland Yard has suggested putting kids who look likely to become criminals in later life on the national DNA register. My natural reaction to this, like most people’s, is revulsion. It really is “like something from a science fiction novel”, and really dark science fiction at that.

The DNA register has some serious problems as it stands, and I haven’t seen these discussed anywhere. The problem is one I have written about before: how hard it is to understand odds when they work at the sorts of levels you encounter with large populations. This sounds really boring but is vitally important to justice.

It is very common now in criminal trial reports to hear that forensic evidence has been a critical part of the conviction. Sometimes a matching DNA sample is the only real evidence, with every other piece of evidence being circumstantial. Odds are quoted by the forensic expert on the stand as being “one in a million” or even a “one in ten million” chance of the sample matching someone else.

These odds sound pretty convincing, and juries certainly find them so. I’ve not heard of any case anywhere where DNA evidence was produced in this manner and the jury found not guilty.

The problem is that these odds are actually not quite so convincing as all that on their own. The argument I’m about to put forward is sometimes called “The Defense Attorney’s Fallacy” because it presumes the only evidence available is the DNA evidence, and that nothing else is available. In most countries there is no such thing as a national register, so the DNA match was found after the suspect was identified by other means. This does make DNA evidence extremely convincing even at quite low odds. This isn’t the case here though - if people are identified by routine DNA sweeps through the database this is most definitely not a fallacy.

Right now anyone who passes through a police station gets their DNA sample taken. Whether they are charged or released that sample is then kept forever. Whenever a serious crime is committed the database is searched for a match. If a match comes up, the police pop over to the home of whoever matches and arrest them.

You’d have to be very lucky not to be charged at this point. A cast-iron alibi would possibly do the job, as would, perhaps, being a High Court Judge or an MP. But perhaps not even then. You are definitely prime suspect, and will probably end up in court, especially if it’s a high profile case with a lot of pressure on the police to arrest someone.

Right now the DNA register has nearly five million records, approaching ten percent of the population. Lets see how well those odds work.

A DNA sample has a “one in ten million” chance of matching someone, say. That means a given sample will match 6 people in the UK, which has a population of sixty million. Ten percent of the population are on the register, roughly, which means that of these 6 the chances are pretty good that one of them is on the register.

This means that for any sample at any crime scene, there will probably be a match with the register - but only a one in six chance that the person who matched actually committed the crime.

This has some pretty far-reaching implications. Imagine if a forensic expert witness instead of quoting a “one in ten million” chance of it being someone else instead said there was an eighty percent chance it was somebody else who did it.

Doesn’t sound so hot now does it?

This is an artifact of the sampling method - if you only sample a random portion of the population your quoted odds have to be modified by the sample rate. This is being completely ignored by everyone in the justice system. They have good reasons for this of course - the police are widely distrusted by juries, and with good reason, since they have such a vested interest in obtaining a conviction. They have finally found a weapon that convinces juries instantly, and the last thing they want to do is undermine it.

Perversely requiring everyone in the country to go on the register might have precisely this effect. For every sample they’d have half a dozen matches, and it might become a lot clearer just how poor odds one in ten million really is, when dealing with populations of the size we are dealing with.

The Government wants ISPs to stop illegal downloads.

This is so wrong headed it’s hard to see what on earth they’re thinking of. Presumably they envisage a technology solution, which is pretty typical amongst people like government ministers - hey well my computer can tell when I’m writing a letter and show me a paperclip, so surely it must be able to stop copyright infringement!

Here’s a few issues that I think are pretty much insoluble:

How can you tell what content infringes copyright?

There’s a couple of options here. Blacklisting known infringers sounds like a good idea, but it’s got problems.

Sites such as The PirateBay don’t themselves distribute copyright material. They host only the (non-copyright) torrent tracker, and the downloaders all share the content amongst themselves.  These downloaders are on moving IP addresses and come and go pretty randomly, and are all over the world and the UK.

So who do you blacklist?

A team of people inspecting torrent sites for suspicious material, and then tracking the torrents, finding the IP addresses of all peers and then adding them to a rolling blacklist that’s used by all IPs might work.  Well work as in generate an actual list of blacklistable IP addresses.  This is the sort of technique the Chinese use, with quite a lot of success.  There will be a lot of collatoral damage though, and in a free society that’s very difficult to justify.  And justify it, in court, they are going to have to do.

Blacklisting The PirateBay sounds good, and is much easier, but new torrent sites will pop up all the time. Again the judiciary will have a very dim view of arbitrary censorship of people who have been convicted of no crime.  I don’t see this method working beyond the first few lawsuits.

Alternatively they might imagine some sort of fingerprinting. Every stream will be examined on the fly, perhaps for the evil bit. With a bit of work it’s possible to probably identify some copyright work using fingerprinting, with quite a few mistakes. Of course, this is defeated utterly by encryption. Right now torrents aren’t encrypted, but I think it will take approximately 1.2 nanoseconds for everyone to move to encrypted torrents if something like this comes in.

Some ISPs might go as far as blocking bittorrent. This is relatively easy to do, and much harder to avoid, however loads of services use bittorrent now that are perfectly legal. Even the BBC’s own iPlayer uses the same sort of technology, and I can see how popular banning that would be.

How do you know it’s working?

They are threatening to punish ISPs unless they “do something”. Precisely how are they going to decide who to punish? Is there going to be some sort of quota - “you have banned 50 users this week, you have unlocked an achievement!”. Sorry, that should be “50 customers”.

There just isn’t a reasonable success metric, and ISPs are not going to voluntarily ban their own customers.  They’ll kick and scream and resist wherever they can, so whatever .gov.uk comes up with is going to have to be enforceable in court.  That means metrics that are clear, fair and measurable.  I just don’t believe such a thing exists.

Ultimately this is just the content exploitation industries failing to address the fact that their business model was temporary. It relied on a particular coincidence of technological limitations and market opportunities. This has changed and now they are about as useful as a bicycle to the proverbial fish.

I just can’t see this happening without a huge amount of damage to the government, and the whole idea being binned in the end.  I just hope they aren’t dumb enough to do it.

Last.FM have announced that they will be providing a huge amount of their catalogue available for free, streamed from their site, with artists paid from advertising and possibly some sort of subscription model.

This is part of a worldwide trend anticipated by many of us for a very long time. Several mobile phone networks are in the process of releasing “music plus” packages, where you get pretty much any music you like, for free, at any time. Again, artists are paid from the phone subscription package.

Obviously streamed music can be copied. Over at Rogue Amoeba, who produce Audio Hijack Pro, they’re an interesting post on this, wondering if this is going to be a problem for the free-streamed model Last.FM have developed.

I don’t think it matters. You won’t bother keeping a copy for yourself for much longer in any case. Why have copies of all those CDs, or MP3s, when it’s all available from the Internet, all the time, at zero cost and effort? The only reason to keep a copy yourself was an artifact of the primitive method of packaging and distribution - not because there being millions of individual copies of a piece of music is inherently useful.

So, in ten year’s time, I reckon the kids won’t have a single copy of mainstream music themselves. Their record collection will consist of a set of bookmarks only - and the whole “music business” as it currently stands will just be a brief “blip” in the history of music, from it’s origins in live-only performance to it’s future as a ubiquitous cultural service in the cloud.

At work we’ve recently had dealings with a web design shop and a huge multinational, both of which were unable to receive files we sent them. The only way to get data to them was to zip it, encrypt the zip and put it on the web. It seems the javascript was enough to send their content filter a bit loopy and it silently refused any emails containing javascript.

Obviously someone somewhere made a decision to block this stuff. Whether they decided correctly is a moot point. The scary thing is the environment that is prompting them to make these decisions. Perhaps 90% of all email is now spam. A large amount of this spam contains malware (evil software), hence, I imagine, the aggressive content filters that gave me so much grief last week.

Facebook was Invented to Stop Spam?

It did lead me to wonder whether these are the dying moments in email interoperation. For all of it’s benefits, email has over the last ten years or so become more and more trouble, and it may become more trouble than it is worth. People are clearly moving to other mediums for their online communication. One of the reasons for the growth in popularity of web forums is that they avoid the grief of handling email (if you can manage to receive the email with the link to confirm your registration of course).

A number of people have told me they use Facebook to communicate with each other because their work email systems think their friend’s emails are spam. Facebook as a spam protection mechanism - just how unwieldy is that.

Of Course, Email is Hard

Internet email has always been more difficult than it looks. The Internet is a complex ecosystem, full of software from different vendors that, although they theoretically follow the same standards, actually have a huge range of behaviours. The Internet worked originally because people were “Tolerant in what they accept” (Postel’s Law), and even in that environment getting a mail server running was non-trivial. These days you would be well advised to make your mailserver as intolerant as you possibly can - only other mailservers that strictly follow the specification should be allowed, in the hope that the worst written are the ones run by spammers.

Furthermore, lots of additional checks are being imposed, from greylisting to multiline banners to pre-greet delays. All of these stretch the specifications a bit, to try to avoid cheaply written ratware. This is a progressing arms race however - as more servers implement these checks the spammers will improve their software to get around it.

The Technical Solutions and Why They Suck

A number of technical means are in progress that attempt to prevent forgery: Domain Keys, SPF, SenderID and DKIM to name but four. A lot of their proponents have claimed these will be an “end to spam”. Unfortunately they will do no good whatsoever. I’m going to quote Rich Kulawiec here, who puts it far better than I.

Problem number one: the bad guys own everyone already

The problem is that we are currently faced with a network environment in which at least 100M systems have been compromised (and some folks, e.g., Vint Cerf, think there are more — his number is 250M)…

Any email access or credentials present on a compromised system are now fully available to its new owner(s). If it has mail privileges by virtue of its network address, they now own those. If it has mail privileges because the user has accounts at (let’s say) their workplace, AOL, and a freemail service, they now own those too. The new owners can send email using the access privileges or credentials at will — either from that system (in the case of network-based privileges, that seems likely) or from another system (username/password pairs) *including* other compromised systems. Note as well that if the compromised system happens to be a mail server, then a large number of credentials may become available to its new owners very rapidly.

And all this email will be passed by any conceivable “anti-forgery” system: it’s coming from “the right” network address range, or it’s using “the right” username/password pair, etc.

– Rich Kulawiec, mailop mailing list, 12.12.2007

Problem number 2: what we do with them when we’ve caught them

Let me try to answer your question this way. Suppose that tomorrow we had in our possession the MAFT (Magical Anti-Forgery Technology) and that it was deployed globally. What happens next?

Well, one thing that happens is that now we have a way to figure out who’s responsible for sending spam (and phishes and whatnot). Okay, so let’s say that we do that, and as a result of that, we identify example.net as a major culprit in, oh, let’s say, mortgage spam. Torrents of it, nonstop, for months on end.

Now what? I’m not being flip, I mean exactly what do we do next?

Some people would say “get them prosecuted” but that’s a non-starter: what they’re doing may not be illegal in some jurisdictions, it’s not considered worthy of much attention, it might take forever, and even then it might not make the spam stop. Other people would say “litigate”, but unless you have very very deep pockets and are prepared to conduct trans-national litigation, forget it. And again, it might not make the spam stop. And so on, down the list of possibilities until we get to: “blacklist them”. Okay, *that* will make the spam stop, and it works immediately. Moreover, nobody’s sanction is necessary for it — we’re all free to stop offering services to anyone at any time for any reason (or none at all). The only people we’re obligated to provide services for are those with a contract for them.

And now we get to the killer problem with this whole line of reasoning, and it’s contained in what I said above:

Well, one thing that happens is that now we have a way to figure out who’s responsible for sending spam (and phishes and whatnot).

*We can do this today.*

We don’t need the MAFT, because we already know who’s responsible for spam — we’ve known for years. It’s whoever’s systems/network are sending it — i.e. this is part of the principle that if it comes from YOUR system/network on YOUR watch then it’s YOURS. This applies whether you run a /32 or a /8.

The problem is not identifying those responsible. Nor is it figuring out who they really are — Spamhaus, SPEWS, Spam-l, NANAE, and numerous other resources have documented this to an amazing level.

The problem is taking effective action once that information is in hand. And the biggest reason the spam problem is as bad as it is today — and will continue to get worse — is that we, collectively, have failed to take effective action. And the only effective action I’ve seen — ever — is blacklisting. Blacklisting is effective because it forces the consequences of the problem back onto the people causing it. Nothing else does that, and of course that’s why everything else — while it might temporarily stop spam — does *nothing* to stop spammers.

– Rich Kulawiec, mailop mailing list, 13.12.2007

This is a recurring problem on the Internet. If you look at fraud, identify theft, credit card theft and all sorts of computer crime the guilty parties are actually well known. If you ask any Internet security researcher they can provide chapter and verse on individuals and organisations who participate in these criminal activities.

Finding the bad guys is not the problem.

The problem is catastrophic failure of law enforcement. Even when Internet crime actually falls within their jurisdiction (unusual) and they have the will to do something about it (virtually unheard of) they are (understandably) woefully clueless about what to actually do about it.

I’ve blogged previously about the Storm Worm and this precise issue, and it applies equally with spam. Rich says that the only thing that works is blacklisting. Unfortunately for blacklisting to really end spam requires a huge number of people to work together, and their actions have unintended consequences - false positives may be acceptable in the wider scheme of things, but they are definitely unacceptable in those specific instances.

A real law enforcement response has to be the ideal solution. It is a very small number of organisations generating this vast quantity of spam - throw a few of them in prison and the quantities would drop rapidly. Catch and punish enough of them and the problem, as it stands now, will end.

I have my doubts about whether this will ever happen though. Email may become a historical oddity as new private forms of communication are adopted that allow people to hide from the
spammers, or that price them out of the market by adding cost. What a shame that would be.

We are the most observed nation in the world. CCTV cameras line our streets, our emails are stored for years as is our IP traffic. Our location is tracked using our mobile phones, and this is stored for years too. Our credit cards record our behaviour and our cash point use correlates our position. If you travel in London, your car is tracked by it’s number plate for congestion charging enforcement and your Oyster card is tracked on every bus, tube and train you use. We have few secrets now.

My instinct is that this is harmful to us as a society, because freedom to choose can only be exercised when unobserved. Our elections are secret ballots for a reason. If you choose to fund animal rights groups, or to go on a demonstration, or to visit a sex shop then these are lawful activities and you should not be prevented from doing them for fear of surveillance. This is not fear of Government necessarily, but there are many people with access to these data, and any of them could be suborned, or could leak the information if they were interested enough. Even celebrities deserve privacy.

Following the horrific story of Madeleine McCann, I kept thinking though that the perpetrators would have been caught by now if this had happened in Britain. Their mobile phones, or their cars, or their faces would have given them away. The police would have issued CCTV pictures within hours. I have young children and the McCann’s are suffering our worst and greatest fear. I would not be human not to be glad that my children are in some ways at least safer precisely because of our level of surveillance.

How to reconcile these? I’d like to propose an idea swiped wholesale from Larry Niven - The Commission for Citizen Oversight. Instituted by Royal Charter and not answerable to Parliament directly, with a self-regulating board of trustees. It’s charter would be to protect citizen’s privacy by storing all of the data deemed personal and private, and they would have a say on what that data is. Mobile phone positioning records, Oyster card records, all output from police and council CCTV cameras. These things would be required by law to be encrypted immediately using the Citizen Oversight public key, and transmitted to their storage facility.

They would have complete discretion in when to release this information to investigating bodies, but would be required by charter to provide data for police investigations into very serious crimes and for reasons of national security. But that’s it. No trolling for celebrities, in fact no trolling at all, or joining up data to invent new suspects.

The British constitution is strangely good at these independent sorts of organisations, that answer only to themselves. In practice it would provide a far greater layer of protection than the disparate, unsecured storage used now that anyone with a mind to could get into (and I expect our own, and other nation’s, security services already have installed back doors in - I know I would if I were them).

This would protect our privacy, would improve national security, and yet would allow the use of the data in instances such as the taking of Madeleine McCann.

Update 20/05/2007.  I’m closing comments on this post now, it’s all got way off topic.

Remember the AACS business last week, where they are threatening anyone who mentions a certain big number with prison. Well, you too can own an integer! That’s right, under the auspices of the well-thought-out DMCA, you too can sue anyone who mentions your number, and just imagine - if every number becomes owned by someone we could end in a world without numbers! Utopia! The DMCA will have finally achieved it’s aims.

Incidentally, E4 DE 37 A0 C7 1F 8B 5A DC F4 F2 C3 6D A4 D8 33 is mine, ALL MINE!  bwahahahahahahahahahahahaha.

This number is illegal.  For those interested, the EFF provide a summary of how the DMCA can make a number illegal.

[Microsoft dropping DRM from Zune Music Store](http://www.computerworld.com/action/article.do?command=viewArticleBasic&taxonomyName=mobile_devices&articleId=9015898&taxonomyId=75). Microsoft have seen the writing on the wall for music DRM much faster than expected, and are [following Apple](http://news.com.com/2100-1027_3-6172398.html?part=rss&tag=2547-1_3-0-5&subj=news) into providing music files without DRM restrictions.

This probably really is the end for music DRM, something that is in the best interests of most music lovers and artists. The revolution that’s finally going to happen in music is going to change the shape of it, and is certainly going to make [some forms of music](http://en.wikipedia.org/wiki/Britney_Spears) uneconomic, but it will also make other forms feasible that once were not. It’s a whole new world out there and if I was an unsigned artist I would be very excited about the possibilities.

What this isn’t is the death of DRM. The impulse that made the music industry commit [such]( http://arstechnica.com/news.ars/post/20050204-4587.html) [public](http://news.bbc.co.uk/1/hi/entertainment/music/3140160.stm) [hara-kiri](http://arstechnica.com/news.ars/post/20060424-6662.html) wasn’t irrational, it was just very poorly judged against their market. In video and television I think we can expect the same behaviour, but with a lot more ferocity. The vast investments in the business models, not least the business models of their suppliers (think Windows Vista), will mean DRM hangs around for some while yet.

Transient content will probably always be advertising supported, since it’s easy and gives access to the most eyeballs. There’s a whole new arms race in there for advertising avoidance software, so the advertising will probably end up pretty subtle. It’ll be just as offensive as TV is now — but hey, go read a book or something.

The DVD market is the one that’s really going to hurt a lot of people. This stuff is being traded big-time on the P2P networks now, and it has become a vital market for a lot of media companies. A lot of series would not be produced without the DVD aftermarket, and the economics of video are, for the moment, different from a lot of music. It will be a few decades before you can make *The West Wing*, from scratch, on your own, in your bedroom, using only a computer. If there isn’t a business model that can support large scale drama with high production values, that would be a real shame.

Video is far less accessible on the move than music too, so being restricted to play your DVDs only on your home player is less of a restriction. I can see a lot of technological battles coming up to try to lock down every single digital and analogue hole in video reproduction. The recent [AACS Crack](http://www.engadget.com/2006/12/27/aacs-drm-cracked-by-backuphddvd-tool/) is only the beginning. Expect a few attempts, some successful, to change the law too.

It looks like [Cricinfo](http://www.cricinfo.com/)’s Cricinfo 3D product probably [stays on the right side of the law](http://www.out-law.com/page-7883). Which is nice. Cricinfo 3D is basically a [machinma](http://en.wikipedia.org/wiki/Machinma) rendition of a live, in-progress, cricket match, based on their own textual commentary. Sky claim that, because the person writing the commentary is watching the TV, therefore the 3D rendition (and presumably the commentary) is in fact a derivative work of their TV broadcast.

What constitutes a derivative work in copyright law is often a pretty hairy question. The concept was designed back when works of creation were pretty simple, but has been constantly tested by the weird and wonderful things people want to make. Certainly the framers of the original concept of “Intellectual Property” would not have considered a machina film application taking an XML internet feed based on text commentary keyed by someone watching satellite television of a cricket match on the other side of the world.

Cricinfo 3D is reminiscent in many ways of [Fanfic](http://en.wikipedia.org/wiki/Fanfic), with the notable exception of the ownership of Cricket itself — which seems to be why Sky are unlikely to win their case against Wisden (publishers of Cricinfo).