a convenient truth

After the ringing endorsement for more “talking rubbish” from Tom in the comments to my last post, I feel newly inspired to spout off.

So, a couple of news items in recent weeks about our government’s incompetent attempts to turn our generally-mostly-well-behaved-as-long-as-you-are-white police force into some kind of robo-stasi.  The ethics of these things are pretty obvious, but what perplexes me is how some of these powers are supposed to be used.

First, the Computer Misuse Act (1995) allows the police to hack into “compromised” systems without a warrant.  Who knew?  Not me. Anyway, apparently they plan to “step up this activity”.

Now as it happens I have briefly met some of the chaps from SOCA, who presumably would be executing this brief.  I am sure they are fine upstanding members of the constabulary, but leet haxors they are not.  Frankly I think it’s unlikely they could drive a pivot table in Excel, let alone devise a 0-day.  The drafters of this act perhaps envisaged the police employing uber hackers from the underground, which superficially sounds quite exciting, but it’s an ITV plot I’m afraid.  If the Old Bill know of uber hackers in the UK they’re most likely to feel their collars.

Alternatively of course they could employ russian hackers, but the amazingly bad idea of involving anyone associated with the FSB with sensitive police business may be apparent even to the clouded minds of our senior officers.

Security firms, on the whole, will also try their best to keep the police off your network, since they won’t be able to tell if it’s the police or not.  For all the fretting about these powers, in practice it’s only those who take no care at all who need to worry, and their machines are probably infested with viruses already.

Second is the rather more disturbing intention of the Government’s to require ISPs to log every email sent. Again, the ethical problems with this are pretty obvious but the practical implications are bizarre.

When you send an email from your workplace to someone else, it’s very likely that your emails never directly touch one of your ISPs mail servers – your mail goes to your corporate mailserver, then over the internet to your receipient’s mailserver.  That mail does traverse your ISPs network, but not their mailservers.

So to log this activity, your ISP would need to run a filter on all TCP traffic for port 25, decode this traffic and extract the headers.  Although this is onerous for ISPs, it’s possible.  It will inevitably make email less reliable, and slower, but hey who cares, right.

But, and this is a but you could drive a truck through, a whole load of people use opportunistic strong encryption for email. It’s enabled out of the box on all decent mail systems these days, and from watching our own logs I guess well more than half of email is encrypted for transport now.

Cracking this is not only difficult-to-impossible, but illegal in many cases. It certainly is more than onerous.

So, may  I just ask, WTF?  Are they really proposing on making laws to legislate for the impossible just to irritate everyone?

Gary Pugh, director of forensic services at Scotland Yard has suggested putting kids who look likely to become criminals in later life on the national DNA register. My natural reaction to this, like most people’s, is revulsion. It really is “like something from a science fiction novel”, and really dark science fiction at that.

The DNA register has some serious problems as it stands, and I haven’t seen these discussed anywhere. The problem is one I have written about before: how hard it is to understand odds when they work at the sorts of levels you encounter with large populations. This sounds really boring but is vitally important to justice.

It is very common now in criminal trial reports to hear that forensic evidence has been a critical part of the conviction. Sometimes a matching DNA sample is the only real evidence, with every other piece of evidence being circumstantial. Odds are quoted by the forensic expert on the stand as being “one in a million” or even a “one in ten million” chance of the sample matching someone else.

These odds sound pretty convincing, and juries certainly find them so. I’ve not heard of any case anywhere where DNA evidence was produced in this manner and the jury found not guilty.

The problem is that these odds are actually not quite so convincing as all that on their own. The argument I’m about to put forward is sometimes called “The Defense Attorney’s Fallacy” because it presumes the only evidence available is the DNA evidence, and that nothing else is available. In most countries there is no such thing as a national register, so the DNA match was found after the suspect was identified by other means. This does make DNA evidence extremely convincing even at quite low odds. This isn’t the case here though – if people are identified by routine DNA sweeps through the database this is most definitely not a fallacy.

Right now anyone who passes through a police station gets their DNA sample taken. Whether they are charged or released that sample is then kept forever. Whenever a serious crime is committed the database is searched for a match. If a match comes up, the police pop over to the home of whoever matches and arrest them.

You’d have to be very lucky not to be charged at this point. A cast-iron alibi would possibly do the job, as would, perhaps, being a High Court Judge or an MP. But perhaps not even then. You are definitely prime suspect, and will probably end up in court, especially if it’s a high profile case with a lot of pressure on the police to arrest someone.

Right now the DNA register has nearly five million records, approaching ten percent of the population. Lets see how well those odds work.

A DNA sample has a “one in ten million” chance of matching someone, say. That means a given sample will match 6 people in the UK, which has a population of sixty million. Ten percent of the population are on the register, roughly, which means that of these 6 the chances are pretty good that one of them is on the register.

This means that for any sample at any crime scene, there will probably be a match with the register – but only a one in six chance that the person who matched actually committed the crime.

This has some pretty far-reaching implications. Imagine if a forensic expert witness instead of quoting a “one in ten million” chance of it being someone else instead said there was an eighty percent chance it was somebody else who did it.

Doesn’t sound so hot now does it?

This is an artifact of the sampling method – if you only sample a random portion of the population your quoted odds have to be modified by the sample rate. This is being completely ignored by everyone in the justice system. They have good reasons for this of course – the police are widely distrusted by juries, and with good reason, since they have such a vested interest in obtaining a conviction. They have finally found a weapon that convinces juries instantly, and the last thing they want to do is undermine it.

Perversely requiring everyone in the country to go on the register might have precisely this effect. For every sample they’d have half a dozen matches, and it might become a lot clearer just how poor odds one in ten million really is, when dealing with populations of the size we are dealing with.

John Gruber, over at Daring Fireball, wonders why there’s no malware for the Mac.  I think they’re mostly there with the reasoning, but they’ve missed a bit.

The reason malware can function so successfully on Windows is because they’re so damn much of it.  Just like diseases that infect humans, there’s a vast range and our antibody system has to be appropriately complex to deal with it.  Virus checkers in windows are pretty complex beasties, and they still only spot 40% of malware.  In effect there is a strong network effect on that platform, encouraging others to join the happy malware ecosystem.

Writing malware for a Mac, right now, would be a no-win situation.  You’d get slapped down so hard you’d probably end up not only with your virus failing to work, but there’d be a worldwide hunt by the mac community to find you.

Being the first virus writer for the Mac has absolutely zero benefits.

At work we’ve recently had dealings with a web design shop and a huge multinational, both of which were unable to receive files we sent them. The only way to get data to them was to zip it, encrypt the zip and put it on the web. It seems the javascript was enough to send their content filter a bit loopy and it silently refused any emails containing javascript.

Obviously someone somewhere made a decision to block this stuff. Whether they decided correctly is a moot point. The scary thing is the environment that is prompting them to make these decisions. Perhaps 90% of all email is now spam. A large amount of this spam contains malware (evil software), hence, I imagine, the aggressive content filters that gave me so much grief last week.

Facebook was Invented to Stop Spam?

It did lead me to wonder whether these are the dying moments in email interoperation. For all of it’s benefits, email has over the last ten years or so become more and more trouble, and it may become more trouble than it is worth. People are clearly moving to other mediums for their online communication. One of the reasons for the growth in popularity of web forums is that they avoid the grief of handling email (if you can manage to receive the email with the link to confirm your registration of course).

A number of people have told me they use Facebook to communicate with each other because their work email systems think their friend’s emails are spam. Facebook as a spam protection mechanism – just how unwieldy is that.

Of Course, Email is Hard

Internet email has always been more difficult than it looks. The Internet is a complex ecosystem, full of software from different vendors that, although they theoretically follow the same standards, actually have a huge range of behaviours. The Internet worked originally because people were “Tolerant in what they accept” (Postel’s Law), and even in that environment getting a mail server running was non-trivial. These days you would be well advised to make your mailserver as intolerant as you possibly can – only other mailservers that strictly follow the specification should be allowed, in the hope that the worst written are the ones run by spammers.

Furthermore, lots of additional checks are being imposed, from greylisting to multiline banners to pre-greet delays. All of these stretch the specifications a bit, to try to avoid cheaply written ratware. This is a progressing arms race however – as more servers implement these checks the spammers will improve their software to get around it.

The Technical Solutions and Why They Suck

A number of technical means are in progress that attempt to prevent forgery: Domain Keys, SPF, SenderID and DKIM to name but four. A lot of their proponents have claimed these will be an “end to spam”. Unfortunately they will do no good whatsoever. I’m going to quote Rich Kulawiec here, who puts it far better than I.

Problem number one: the bad guys own everyone already

The problem is that we are currently faced with a network environment in which at least 100M systems have been compromised (and some folks, e.g., Vint Cerf, think there are more — his number is 250M)…

Any email access or credentials present on a compromised system are now fully available to its new owner(s). If it has mail privileges by virtue of its network address, they now own those. If it has mail privileges because the user has accounts at (let’s say) their workplace, AOL, and a freemail service, they now own those too. The new owners can send email using the access privileges or credentials at will — either from that system (in the case of network-based privileges, that seems likely) or from another system (username/password pairs) *including* other compromised systems. Note as well that if the compromised system happens to be a mail server, then a large number of credentials may become available to its new owners very rapidly.

And all this email will be passed by any conceivable “anti-forgery” system: it’s coming from “the right” network address range, or it’s using “the right” username/password pair, etc.

– Rich Kulawiec, mailop mailing list, 12.12.2007

Problem number 2: what we do with them when we’ve caught them

Let me try to answer your question this way. Suppose that tomorrow we had in our possession the MAFT (Magical Anti-Forgery Technology) and that it was deployed globally. What happens next?

Well, one thing that happens is that now we have a way to figure out who’s responsible for sending spam (and phishes and whatnot). Okay, so let’s say that we do that, and as a result of that, we identify example.net as a major culprit in, oh, let’s say, mortgage spam. Torrents of it, nonstop, for months on end.

Now what? I’m not being flip, I mean exactly what do we do next?

Some people would say “get them prosecuted” but that’s a non-starter: what they’re doing may not be illegal in some jurisdictions, it’s not considered worthy of much attention, it might take forever, and even then it might not make the spam stop. Other people would say “litigate”, but unless you have very very deep pockets and are prepared to conduct trans-national litigation, forget it. And again, it might not make the spam stop. And so on, down the list of possibilities until we get to: “blacklist them”. Okay, *that* will make the spam stop, and it works immediately. Moreover, nobody’s sanction is necessary for it — we’re all free to stop offering services to anyone at any time for any reason (or none at all). The only people we’re obligated to provide services for are those with a contract for them.

And now we get to the killer problem with this whole line of reasoning, and it’s contained in what I said above:

Well, one thing that happens is that now we have a way to figure out who’s responsible for sending spam (and phishes and whatnot).

*We can do this today.*

We don’t need the MAFT, because we already know who’s responsible for spam — we’ve known for years. It’s whoever’s systems/network are sending it — i.e. this is part of the principle that if it comes from YOUR system/network on YOUR watch then it’s YOURS. This applies whether you run a /32 or a /8.

The problem is not identifying those responsible. Nor is it figuring out who they really are — Spamhaus, SPEWS, Spam-l, NANAE, and numerous other resources have documented this to an amazing level.

The problem is taking effective action once that information is in hand. And the biggest reason the spam problem is as bad as it is today — and will continue to get worse — is that we, collectively, have failed to take effective action. And the only effective action I’ve seen — ever — is blacklisting. Blacklisting is effective because it forces the consequences of the problem back onto the people causing it. Nothing else does that, and of course that’s why everything else — while it might temporarily stop spam — does *nothing* to stop spammers.

– Rich Kulawiec, mailop mailing list, 13.12.2007

This is a recurring problem on the Internet. If you look at fraud, identify theft, credit card theft and all sorts of computer crime the guilty parties are actually well known. If you ask any Internet security researcher they can provide chapter and verse on individuals and organisations who participate in these criminal activities.

Finding the bad guys is not the problem.

The problem is catastrophic failure of law enforcement. Even when Internet crime actually falls within their jurisdiction (unusual) and they have the will to do something about it (virtually unheard of) they are (understandably) woefully clueless about what to actually do about it.

I’ve blogged previously about the Storm Worm and this precise issue, and it applies equally with spam. Rich says that the only thing that works is blacklisting. Unfortunately for blacklisting to really end spam requires a huge number of people to work together, and their actions have unintended consequences – false positives may be acceptable in the wider scheme of things, but they are definitely unacceptable in those specific instances.

A real law enforcement response has to be the ideal solution. It is a very small number of organisations generating this vast quantity of spam – throw a few of them in prison and the quantities would drop rapidly. Catch and punish enough of them and the problem, as it stands now, will end.

I have my doubts about whether this will ever happen though. Email may become a historical oddity as new private forms of communication are adopted that allow people to hide from the
spammers, or that price them out of the market by adding cost. What a shame that would be.

We have a massive problem with the Internet.  The massive penetration of malware has reached epidemic proportions, and it’s hard to see how to fix it.  This PDF has some great slides that show how the malware industry works.
Everyone you ask will have a different target to blame: Microsoft, application vendors, insecure protocols and standards, the police, clueless users .  The real problem is a network effect – it really takes a combination of failures to make this problem as gigantic as it now is.  There is a real risk of the end of the Internet as we know it.

A good example of this is the Storm Worm.  When this hits, we could see the largest piece of military or economic infowar ever undertaken, presumably depending on who they auction their network to.  Seriously, this is going to be huge.

Unless the security community take their responsibilities seriously and combat this directly, it’s hard to know how the Internet can cope with such widespread infection.  However the state-sponsored police organisations are woefully clueless, and the guys who know what to do are paralysed by fear of prosecution, fear of making a mistake, and fear of execution by Russian hit-men.  Seriously.  If this was a movie, you wouldn’t believe it.

We are the most observed nation in the world. CCTV cameras line our streets, our emails are stored for years as is our IP traffic. Our location is tracked using our mobile phones, and this is stored for years too. Our credit cards record our behaviour and our cash point use correlates our position. If you travel in London, your car is tracked by it’s number plate for congestion charging enforcement and your Oyster card is tracked on every bus, tube and train you use. We have few secrets now.

My instinct is that this is harmful to us as a society, because freedom to choose can only be exercised when unobserved. Our elections are secret ballots for a reason. If you choose to fund animal rights groups, or to go on a demonstration, or to visit a sex shop then these are lawful activities and you should not be prevented from doing them for fear of surveillance. This is not fear of Government necessarily, but there are many people with access to these data, and any of them could be suborned, or could leak the information if they were interested enough. Even celebrities deserve privacy.

Following the horrific story of Madeleine McCann, I kept thinking though that the perpetrators would have been caught by now if this had happened in Britain. Their mobile phones, or their cars, or their faces would have given them away. The police would have issued CCTV pictures within hours. I have young children and the McCann’s are suffering our worst and greatest fear. I would not be human not to be glad that my children are in some ways at least safer precisely because of our level of surveillance.

How to reconcile these? I’d like to propose an idea swiped wholesale from Larry Niven – The Commission for Citizen Oversight. Instituted by Royal Charter and not answerable to Parliament directly, with a self-regulating board of trustees. It’s charter would be to protect citizen’s privacy by storing all of the data deemed personal and private, and they would have a say on what that data is. Mobile phone positioning records, Oyster card records, all output from police and council CCTV cameras. These things would be required by law to be encrypted immediately using the Citizen Oversight public key, and transmitted to their storage facility.

They would have complete discretion in when to release this information to investigating bodies, but would be required by charter to provide data for police investigations into very serious crimes and for reasons of national security. But that’s it. No trolling for celebrities, in fact no trolling at all, or joining up data to invent new suspects.

The British constitution is strangely good at these independent sorts of organisations, that answer only to themselves. In practice it would provide a far greater layer of protection than the disparate, unsecured storage used now that anyone with a mind to could get into (and I expect our own, and other nation’s, security services already have installed back doors in – I know I would if I were them).

This would protect our privacy, would improve national security, and yet would allow the use of the data in instances such as the taking of Madeleine McCann.

Update 20/05/2007.  I’m closing comments on this post now, it’s all got way off topic.

[Travelling with expensive cameras](http://www.schneier.com/blog/archives/2006/09/expensive_camer.html), from Bruce Schneier.