a convenient truth

I got an Eee PC just before christmas.  I’ve used it for a few days, so here are some first impressions.

First, the good: the build quality is good, the screen is excellent, the sound is good as is the microphone.  The webcam works fine, the touch pad is well tuned and the OS it ships with is very nicely put together. The wifi worked out of the box with WPA, which is better than I can say for any laptop I’ve used before, with any OS.

There’s a lot of talk about how to put Windows XP on them.  I honestly can’t see the point.

In fact the whole package is ludicrously good for it’s price point.  So good I think Asus have really invented a whole new category (Charlie Stross has some well-considered thoughts on this).  Any PC manufacturer who isn’t now planning their own line in this new category is an idiot.

There have been “Ultra Micro” PCs before, but the price is a key factor - this is the first disposable general purpose computer.  There’s no point fretting about the lifetime of the integral solid state drive, because you will have bought a new eee pc before it runs out.  I’m already planning to get another one (and this time it’ll be black).

One of the things you notice about new technology categories is that you find new use cases you didn’t know you had.  I’ve started listening to podcasts in bed - the eee functions as a bedside internet radio.  Last night, the Now Show. The night before, In Our Time.  Honestly, it’s worth getting one just for bedtime podcasts.

There are some downsides.  The 800×480 screen is not big enough for a number of websites.  I hope this encourages developers to consider small screens when building sites - I’ll certainly be testing our new sites on the eee  myself.  There is enough physical room in the lid of the machine for a larger screen, and I bet that’s near the top of their list for new models.

Not as high, I hope, as bluetooth. This is the only real gripe  I have (apart from a perennial one below).  Bluetooth is so ubiquitous I was really surprised to see it absent.  It’s amazingly useful for mobile Internet, something I use quite a bit anyway (even if mostly with train or coffee shop wifi).

You can plug in a bluetooth dongle into one of the three (3!) USB ports of course, but it’s nowhere near as neat.  This machine really needs bluetooth.  Obviously 3G, when it comes, will be even better.  As soon as they release a machine with integral 3G, I’m getting it.

Other models will obviously have more RAM and a larger disk, but the RAM is field upgradable and the disk is actually large enough for most people, even with only 4GB.  Adding an SDHC card for storage is trivial, and cheap.  Play.com do a 16GB one for 50 quid.

My one other gripe? The keyboard is designed for right handers.  The space bar doesn’t work if you tap it on the left.  If you are a left-handed typist you hit the space bar with your left thumb.  I’ve had to change my hand position on the otherwise perfectly usable keyboard so I can hit space.

Us left-handers are a sizable minority of the population.  We are honestly worth considering when designing stuff.  Saying “ah well 10% of our users are going to have a substandard experience because we cannot be bothered to support them” is NOT good enough.  Asus are nowhere near alone in this though.  Perhaps fuel for a future post.

All that said, I’m writing this post on the Eee, and it is more than acceptable.  This really is one of the best designed bits of tech I’ve got in many many years.  You should seriously consider buying one, you are bound to use it more than you can imagine.

At work we’ve recently had dealings with a web design shop and a huge multinational, both of which were unable to receive files we sent them. The only way to get data to them was to zip it, encrypt the zip and put it on the web. It seems the javascript was enough to send their content filter a bit loopy and it silently refused any emails containing javascript.

Obviously someone somewhere made a decision to block this stuff. Whether they decided correctly is a moot point. The scary thing is the environment that is prompting them to make these decisions. Perhaps 90% of all email is now spam. A large amount of this spam contains malware (evil software), hence, I imagine, the aggressive content filters that gave me so much grief last week.

Facebook was Invented to Stop Spam?

It did lead me to wonder whether these are the dying moments in email interoperation. For all of it’s benefits, email has over the last ten years or so become more and more trouble, and it may become more trouble than it is worth. People are clearly moving to other mediums for their online communication. One of the reasons for the growth in popularity of web forums is that they avoid the grief of handling email (if you can manage to receive the email with the link to confirm your registration of course).

A number of people have told me they use Facebook to communicate with each other because their work email systems think their friend’s emails are spam. Facebook as a spam protection mechanism - just how unwieldy is that.

Of Course, Email is Hard

Internet email has always been more difficult than it looks. The Internet is a complex ecosystem, full of software from different vendors that, although they theoretically follow the same standards, actually have a huge range of behaviours. The Internet worked originally because people were “Tolerant in what they accept” (Postel’s Law), and even in that environment getting a mail server running was non-trivial. These days you would be well advised to make your mailserver as intolerant as you possibly can - only other mailservers that strictly follow the specification should be allowed, in the hope that the worst written are the ones run by spammers.

Furthermore, lots of additional checks are being imposed, from greylisting to multiline banners to pre-greet delays. All of these stretch the specifications a bit, to try to avoid cheaply written ratware. This is a progressing arms race however - as more servers implement these checks the spammers will improve their software to get around it.

The Technical Solutions and Why They Suck

A number of technical means are in progress that attempt to prevent forgery: Domain Keys, SPF, SenderID and DKIM to name but four. A lot of their proponents have claimed these will be an “end to spam”. Unfortunately they will do no good whatsoever. I’m going to quote Rich Kulawiec here, who puts it far better than I.

Problem number one: the bad guys own everyone already

The problem is that we are currently faced with a network environment in which at least 100M systems have been compromised (and some folks, e.g., Vint Cerf, think there are more — his number is 250M)…

Any email access or credentials present on a compromised system are now fully available to its new owner(s). If it has mail privileges by virtue of its network address, they now own those. If it has mail privileges because the user has accounts at (let’s say) their workplace, AOL, and a freemail service, they now own those too. The new owners can send email using the access privileges or credentials at will — either from that system (in the case of network-based privileges, that seems likely) or from another system (username/password pairs) *including* other compromised systems. Note as well that if the compromised system happens to be a mail server, then a large number of credentials may become available to its new owners very rapidly.

And all this email will be passed by any conceivable “anti-forgery” system: it’s coming from “the right” network address range, or it’s using “the right” username/password pair, etc.

– Rich Kulawiec, mailop mailing list, 12.12.2007

Problem number 2: what we do with them when we’ve caught them

Let me try to answer your question this way. Suppose that tomorrow we had in our possession the MAFT (Magical Anti-Forgery Technology) and that it was deployed globally. What happens next?

Well, one thing that happens is that now we have a way to figure out who’s responsible for sending spam (and phishes and whatnot). Okay, so let’s say that we do that, and as a result of that, we identify example.net as a major culprit in, oh, let’s say, mortgage spam. Torrents of it, nonstop, for months on end.

Now what? I’m not being flip, I mean exactly what do we do next?

Some people would say “get them prosecuted” but that’s a non-starter: what they’re doing may not be illegal in some jurisdictions, it’s not considered worthy of much attention, it might take forever, and even then it might not make the spam stop. Other people would say “litigate”, but unless you have very very deep pockets and are prepared to conduct trans-national litigation, forget it. And again, it might not make the spam stop. And so on, down the list of possibilities until we get to: “blacklist them”. Okay, *that* will make the spam stop, and it works immediately. Moreover, nobody’s sanction is necessary for it — we’re all free to stop offering services to anyone at any time for any reason (or none at all). The only people we’re obligated to provide services for are those with a contract for them.

And now we get to the killer problem with this whole line of reasoning, and it’s contained in what I said above:

Well, one thing that happens is that now we have a way to figure out who’s responsible for sending spam (and phishes and whatnot).

*We can do this today.*

We don’t need the MAFT, because we already know who’s responsible for spam — we’ve known for years. It’s whoever’s systems/network are sending it — i.e. this is part of the principle that if it comes from YOUR system/network on YOUR watch then it’s YOURS. This applies whether you run a /32 or a /8.

The problem is not identifying those responsible. Nor is it figuring out who they really are — Spamhaus, SPEWS, Spam-l, NANAE, and numerous other resources have documented this to an amazing level.

The problem is taking effective action once that information is in hand. And the biggest reason the spam problem is as bad as it is today — and will continue to get worse — is that we, collectively, have failed to take effective action. And the only effective action I’ve seen — ever — is blacklisting. Blacklisting is effective because it forces the consequences of the problem back onto the people causing it. Nothing else does that, and of course that’s why everything else — while it might temporarily stop spam — does *nothing* to stop spammers.

– Rich Kulawiec, mailop mailing list, 13.12.2007

This is a recurring problem on the Internet. If you look at fraud, identify theft, credit card theft and all sorts of computer crime the guilty parties are actually well known. If you ask any Internet security researcher they can provide chapter and verse on individuals and organisations who participate in these criminal activities.

Finding the bad guys is not the problem.

The problem is catastrophic failure of law enforcement. Even when Internet crime actually falls within their jurisdiction (unusual) and they have the will to do something about it (virtually unheard of) they are (understandably) woefully clueless about what to actually do about it.

I’ve blogged previously about the Storm Worm and this precise issue, and it applies equally with spam. Rich says that the only thing that works is blacklisting. Unfortunately for blacklisting to really end spam requires a huge number of people to work together, and their actions have unintended consequences - false positives may be acceptable in the wider scheme of things, but they are definitely unacceptable in those specific instances.

A real law enforcement response has to be the ideal solution. It is a very small number of organisations generating this vast quantity of spam - throw a few of them in prison and the quantities would drop rapidly. Catch and punish enough of them and the problem, as it stands now, will end.

I have my doubts about whether this will ever happen though. Email may become a historical oddity as new private forms of communication are adopted that allow people to hide from the
spammers, or that price them out of the market by adding cost. What a shame that would be.

I can’t put it better than Mike does, really. Ordnance Survey have released our data a little bit, and it’s very welcome.  What they haven’t done is make it available for commercial use.

Since we paid for the collection of data in the first place, and since there is such an obvious vast benefit to the residents of the UK in making this data freely available, I find their continued insistence on vast license fees frankly incomprehensible.

We have a massive problem with the Internet.  The massive penetration of malware has reached epidemic proportions, and it’s hard to see how to fix it.  This PDF has some great slides that show how the malware industry works.
Everyone you ask will have a different target to blame: Microsoft, application vendors, insecure protocols and standards, the police, clueless users .  The real problem is a network effect - it really takes a combination of failures to make this problem as gigantic as it now is.  There is a real risk of the end of the Internet as we know it.

A good example of this is the Storm Worm.  When this hits, we could see the largest piece of military or economic infowar ever undertaken, presumably depending on who they auction their network to.  Seriously, this is going to be huge.

Unless the security community take their responsibilities seriously and combat this directly, it’s hard to know how the Internet can cope with such widespread infection.  However the state-sponsored police organisations are woefully clueless, and the guys who know what to do are paralysed by fear of prosecution, fear of making a mistake, and fear of execution by Russian hit-men.  Seriously.  If this was a movie, you wouldn’t believe it.

There is something horrific about this. The Israeli Defence Force, which is probably the most competent army in the world (recent cock ups notwithstanding) have philosophers in their ranks.  And the stuff they are reading and applying is weird.  You would not like to be anywhere near these guys when they execute their theories in hardware:

We read Christopher Alexander, can you imagine?; we read John Forester, and other architects. We are reading Gregory Bateson; we are reading Clifford Geertz. Not myself, but our soldiers, our generals are reflecting on these kinds of materials. We have established a school and developed a curriculum that trains “operational architects”.’4 In a lecture Naveh showed a diagram resembling a ‘square of opposition’ that plots a set of logical relationships between certain propositions referring to military and guerrilla operations. Labelled with phrases such as ‘Difference and Repetition – The Dialectics of Structuring and Structure’, ‘Formless Rival Entities’, ‘Fractal Manoeuvre’, ‘Velocity vs. Rhythms’, ‘The Wahabi War Machine’, ‘Postmodern Anarchists’ and ‘Nomadic Terrorists’, they often reference the work of Deleuze and Guattari. War machines, according to the philosophers, are polymorphous; diffuse organizations characterized by their capacity for metamorphosis, made up of small groups that split up or merge with one another, depending on contingency and circumstances. (Deleuze and Guattari were aware that the state can willingly transform itself into a war machine. Similarly, in their discussion of ‘smooth space’ it is implied that this conception may lead to domination.)

I’m not sure I’ve ever heard such bizarre theoretical justification for murder.

My arse.  via mefi.

Everybody gets bouts of existential angst, and each generation seems to invent it’s own. For the last few generations the focus of that fear was mostly Mutually Assured Destruction by nuke. Now the environment seems a worthy subject to fret about.

There is a bigger subject to worry about though, if you feel you lack sufficient angst. It’s a little theoretical and geeky, but what the hell.

It’s called the Fermi Paradox. Over lunch in 1950 one day the physicist Enrico Fermi was discussing the lack of evidence of extraterrestrial civilizations with colleagues. At one point he asked “Where is everybody?“. This has come to be known as the Fermi Paradox. Become an important physicist and even random lunchtime rants become Capitalized.

Initially it was thought of as a pretty silly question, but over the years it has gained in significance. We have gathered a vast amount of evidence about the universe and two things seem pretty clear:

1. Intelligent life ought to be everywhere. We’ve got a vast, ancient universe full of niches that could occupy intelligent life.
2. We can’t find any

So. Where the hell are they? There are a whole bunch of possible explanations, although obviously we can’t know which is true. The reason for the angst is that few of the explanations are very nice.

Here’s a selection:

God: God exists. He created the universe specifically for mankind for ineffable reasons of his own. We are the pawns of an omnipotent and omniscient being whose purpose we cannot even guess, and He chose not to create extraterrestrials. The stars may not even be real, but just painted on the inside of the Heavenly Firmament. Even death may be no escape from His grasp.

The Simulation Argument: Computers will continue to gain in power until it is easy to run an entire simulation of a planet, at a detail sufficient to provide consciousness (or the illusion of it) to parts of the simulation. In such a world, may millions of organisations may choose to run simulations for their own purpose. In this world, the odds against us being in the real universe are pretty slim - in fact this is almost certainly a sim. c.f. God.

The Weak Anthropic Principle: Actually, for some reason we don’t know, intelligent life is vanishingly unlikely. The vast majority of possible universes are sterile spaces, vast orrerys unpolluted by life. Ours would be, except we happen to be here to observe it which is purely chance - after all, we have to be here to observe it.

Life is hard: Many many intelligent lifeforms have approached civilization only to be destroyed. The combination of resource usage and the advent of mass death weapons proves too challenging for almost all life, and they die out. We too will probably become extinct through our own actions.

Ascendency: There is a spiritual element to the universe unknown by the majority of us. Intelligent life is rife, but with awareness of this spiritual sphere they have ascended there to dwell in whatever marvels it provides.

Singularity: We are approaching a point of technological singularity where the rate of change becomes asymptotic and ‘takes off’. At this point we can make no predictions beyond the singularity. Every intelligent civilization reaches this point, and we cannot know what happens next.

Resource extinction: There have been starfaring civilizations, however competition for resource is intense and few have survived. Most solar systems are surrounded by the looted wrecks of planets. They haven’t reached us yet, but when they do we will be obliterated.

Upload society: The technology required to run a sentient being on different hardware turns out to be pretty easy. By uploading yourself to virtual communities you gain immortality and massive control over your environment. As resources dwindle this becomes the default choice. There are vast numbers of intelligent civilizations, but they are resident in nanoscale computers, living complex and ineffable lives that we cannot perceive.

They are hiding: They know we are here, and are watching us. For their own reasons they are hiding, perhaps in collusion with shadowy terrestrial government agents.

I am sure there are some more I’ve missed too, probably including the real explanation. It could of course be a mixture of all of them. All of those explanations send a shiver down my spine though, although some seem nicer than others.

Well, a specific heavy metal: Lead.

“Round about the turn of the century many members of the western world would rather shit themselves than go to the gym. For real”.