uberconcept.com

I have a lot of passwords. Before I even walk out the door of my home or acess anything online, I have passwords for servers, desktops, apps on servers etc, probably 20 passwords. Add in the rest and I’d have over 200. Add in enforced password change policies and it gets worse. Clearly you can’t remember all of these.

There are generally two ways of managing this sort of thing.

1. Re-use passwords

One solution is to have a small number of known passwords. This is generally a bad idea as compromising one password means compromising another. This is a security problem both at your end and the destination. At your end somone may find out a password (eg by shoulder surfing) or you give it to someone (eg login to my yahoo account and download this for me).

At the far end, the site or system you have the password may have lax security. They might be storing the passwords using clear text rather than hashed and salted. If the site is compromised, your password might be compromised with it. Worse, the site might be set up simply to collect logins and passwords.

This can be mitigated by using high value passwords with high value sites/systems, effectively assessing their value before giving them a password. For example your bank account would be a high value password, your email account would be a medium value password and a web forum might be a low value password. So you might have one password for each area.

There are two problems with this approach. This will still mean that one compromise of your high value password will mean all your high value systems are compromised. Using different passwords for each high value system may not be an option if you have a large number of high value systems. The other problem is you may not be able to make a good assessment as to the value of a system. For example, your email account might seem to be relatively low value, however most sites offer the option to reset passwords with emails notifications. Hence compromising you email account can guarantee the compromise of a lot of other systems.

2. Password storage

The other option is to use a password storage/management system. These store the passwords in an encrypted datastore, with a password to access the datastore. To use this properly you need to have a strong password for the datastore, as cracking that password gives someone full access to your passwords everywhere. You also would not want to forget that one strong password, or lose the datastore. In either case you are in trouble.

One variation on this is to use different datastores with different passwords for different areas (eg work, net accounts, bank accounts etc). This allows you to limit the potential damage of that password being compromised, but also increases the liability of having multiple passwords you must remember.

This is a much better solution than re-using passwords as it significantly reduces both the password management headache and also reduces the risk of a single password loss resulting multiple systems being compromised. It does increase the risk of compromise from that single password, however carefully managed, the overall risk is lowered

Last October I finally cracked and got an iphone 3Gs.

Part of the reason for this was my ageing palm pilot wouldn’t sync with Window 7. I also needed a new phone. I decided against an Android based phone as I just wasn’t convinced that it would be as user friendly, it also seemed to have a less healthy developer ecosystem than the iPhone.

The iPhone ticked the following boxes:

• Personal organiser - calender, notes, TODO, contacts, syncing up to the web
• Tethering -plug into the laptop for net access on the go
• iPod -I didn’t have one, was thinking of getting one
• Internet device - for net and email access anywhere
• Phone -
• App Store - if it doesn’t have it out of the box I can probably write or buy something to do it

Having owned it for ~6 months I’m still enthused by it. I think this has done more than any other single tech purchase to make me more optimistic about new technology. It it quite simply one of the best hardware software combinations I’ve ever come across.

I think the best illustration of the ease of use is three anecdotes. I gave the phone to someone who had never used one with no instructions while driving so they could provide directions. They could used it immediately. While out at dinner one night I saw one 70 year old showing another 70 year old photos on her iphone. I wouldn’t have called 70 year old ladies a great market for high technology. Lastly, in the 6 months I’ve had it I’ve only opening a manual one and that was to find out how to undo (as it turns out shake).

Out of the box it is a great, add the app store and you have something really special. I found the note taking app pretty anemic, particularly as it couldn’t sync to the web, so I picked up Simple Note. The lack of a todo list was rectified by Toodledo. I even found the Good Food Guide was available as an app. And then there were all the games…

I think the app store is one of the best examples of what can happen when you create a great platform (the iPhone) and make development relatively friendly. I also thing the walled garden aspect of the app store is a net positive for consumers as it helps filter out a lot of noise.

It has been a while between posts, apologies I’ve been busy, especially with work. I also ran into a bug in wordpress as I tried to log in to create a post. Essentially if you logged in with admin rights but were not an admin, and you needed to upgrade the install, it endlessly redirected.

So a quick tech summary of the last year, which I will expand on in further posts:

• iPhone - I got one, it is great, highly recommended
• Snow Leopard - I finally moved my laptop to snow leopard, seems a bit better
• Windows 7 - I ran the RC for 8 months or so, looks good, just installed the RTM version on my desktop
• Virtualisation - I’ve got very excited about virtualisation in the last 6 months, particularly vmware
• Code - I’ve found a language worse than perl: Powerbuilder
• Library project in Python - I found Library thing and decided I didn’t have enough time to write my own

Apologies for the site being offline for ~5 weeks. It is a long story which I will fill in later, but the short version is that is the last 2 months I’ve moved jobs and moved house. The moving house part involved taking me offline, which meant taking various servers offline including this one.

While looking at a new flash drive, I spent some time looking again at PortableApps. In case you’ve never seen this, portable apps are basically apps optimised to run from your flash drive. So you can walk up to another computer, plug in your flash drive and feel right at home. I’d always assumed that PortableApps would be stateless and that settings would not be remembered between uses. I was wrong.

The first discovery was just how well thunderbird works from PortableApps. I run my own mail server, with remote access via IMAP over SSL. I have installed squirrelmail, but I’ve found it to be generally pretty slow on my low end hardware. So being able to plug in a flash drive and to be able to just run thunderbird for IMAP over SSL solves an immediate problem.

I was happy to discover that portable apps also quite happily allows you to install plugins for thunderbird and firefox. This is a great, I generally have about 10 plugins installed for firefox. For example, this means I can have bookmarks synchronised with foxmarks.

Ideally for something like this, you want to have all the mail cached locally so there isn’t too much time spent loading up messages. However if I do this and I lose the drive, someone would have all my emails. Even if I did not cache all my emails, they would still have cached the headers, which is far too much information.

Enter TrueCrypt, true friend of the appropriately paranoid. TrueCrypt is an encryption program that can encrypt whole drives. It includes a traveller mode for flash drives, just what I was looking for. Using TrueCrypt, you can encrypt the entire PortableApps folder, ensuring that if you lose your drive your data is still safe. Unfortunately it seems to require admin privileges, but in most cases that should be possible to arrange.

I haven’t bought the new drive yet, I’ll report any issues when I get it.

When I wake up I normally can’t get to sleep, regardless of what time it is.

This happened recently when I was helping an organisation test a web application. I woke up at about 1am and couldn’t get back to sleep, by about 5am I’d tested everything and written out a full report for them.

This morning I couldn’t sleep again, so I worked on solving what has been a long running problem for me.

I was looking for a flash drive to replace my current rather battered 1GB drive. Once my flash drive sat happily on my keyring. First the keyring connector broke. Then the plastic casing came apart and I was left with this:

It works just fine, sits in a jeans pocket well and has been through the wash safely at least half a dozen times. However I do leave it behind at times so I really want something that I can put on a keyring. The other problem is that I keep finding myself running short of space.

Unfortunately most flash drives have pretty fragile keyring links. There are some that don’t, but they seem to by ultra-rugged style drives that would be rather heavy on a keyring. Additionally, I don’t see the point of waterproofing flash drives. If mine has been through the laundry half a dozen times quite safely, what is the point? So long as it dry inside and out when I plug it in it should work just fine.

Fortunately I found the SanDisk Cruzer Titanium. Not too large, tough exterior and a good strong keyring link.

So unfortunately I haven’t had as much time as I would have liked to work on my book library project. What with buying a unit, Christmas, going to NZ and helping test a software app on the side, I just haven’t had the time.

However I had made some progress.

I’ve done some basic work familiarising myself with python. I’ve come to the conclusion that it would have to be one of the best languages I’ve ever worked with. The only way I can describe it is as if C++ and perl reproduced and had a child that took the best features from both languages and none of the worst features. Glee! It is very easy to write nice clean code that makes sense.

I also bought a CueCat for scanning the barcodes. My reasoning was that it was cheap and was likely to have good support on many platforms since there are so many of them around. I’ve since found that it can be little slow to scan barcodes, but is certainly good enough for the moment. I’ve ported some code to decode the CueCat output from javascript to python.

My original plan was to build a GUI for this using wxpython. Since then I’ve discovered that the CueCat doesn’t need complicated drivers, it just dumps encoded output similar to the way a keyboard does. So there is no real need for a heavy client on a desktop, I can just skip to building the whole thing as a web application.

I’ve currently looking at different frameworks, but at the moment django looks pretty sweet.

Vic and I just got back from a great trip to New Zealand.

We had a lovely time there and took a lot of photos.

I’ve decided to start a new project, and like all good home projects, at scratches an itch. I’ve also wanted to teach myself a new language or two and the best way to do that is to write some code.

I read a lot of books, I’m pretty omnivorous so I’ll happily read history, science fiction, fantasy, fiction and older literature. I also like to own books so that I can re-read them or refer to them later. So I have a lot of books.

I’m starting to run into problems organising all my books, I forget which books I have and where they are.

Obviously this calls for a database, but I have no inclination to type in the author, title, publication date, ISBN etc into a database for the (est) thousands of books I have.

So here is the project. I want to build a tool that can use a barcode scanner to retrieve the ISBN from books, query an online database and do some operations on the database. Operations would include, add book, move book, get rid of book. In addition the list of books will be published in a website, possibly with some filtering tools

I was chatting to a friend about this and he suggested that this could be great for managing CDs and DVDs. That might make an interesting extension of the project.

Now I’m pretty sure that I could find something that does most of this, but I have been looking for a project to work on for a while and this looks like fun. I’ll release the whole thing under GPL3 and possibly host it on source force if it gets to a polished enough state.

The current plan is:

1. Learn python, my new language of choice (in progress)
2. Work out how to get barcode reader to work and retrieve ISBN from barcode. This is pretty pivotal to the project so I should work that out first
3. Find an appropriate online store to query for the details on the book. Amazon is one option but I need to check out their ToS.
4. Design the database, I’ll aim to make it as cross platform as possible, but probably using mysql
5. Build the class structure
6. Build the GUI interface for adding books. Would also need a manual ISBN entry option, not all books have barcodes. Current thinking favors using wxPython for this
7. Build the web interface
8. Install/build scripts.

I’m really looking forward to getting my teeth into this. I’ll add updates as this progresses.

Spam is a problem of enormous proportions. Current estimates figure that over 80% of all email is spam.

Some time ago I wrote a post about some changes to the configuration of my mail server that cut down the spam drastically. I thought I might take a moment to talk about the various techniques that are used to combat spam.

Some terminology I’m going to use:

• spam - unwanted email
• ham - wanted email
• false positive - ham that is marked as spam
• client - mail client, eg Thunderbird, Outlook
• server - mail server, eg Exchange, postfix
• host - someone who hosts servers
• Joe Job - when spam is sent using the email address of someone else

Bayesian Filtering

This originated from Paul Graham. The idea was that you break a message up into tokens and then examine the tokens against a database of tokens. Each of the tokens in your database has a score as to how spammy the token is. The individual scores are combined to provide a score for an email. Emails are then rejected or allowed based on that score. This requires that you train your filter on collections of spam and ham.

Spammer responses

• replacing letters with numbers (v1arga) or adding in spaces. This is generally pretty ineffective.
• Attempt to poison the filters with random text
• Delivering their payload as an image

Advantages

• Generally cuts spam significantly (>75%)
• Can be configured and trained to specific needs
• Can be run on the client (eg Thunderbird) or the server

Disadvantages

• CPU intensive, a burden borne by the receiver of the email.
• Doesn’t tend to scale well, over an organisation. One person’s spam is another person’s ham.

Realtime Black List (RBL)

A RBL works by storing a known list of IP addresses or IP address blocks that send spam. When a server receives a HELO request, it checks the IP address of the sender against the RBL. If the IP address matches a known spammer IP address, it refuses the email. One issue with RBLs is that they are often easy to get on to and hard to get off. In addition some RBLs take the view that if even if just a single IP address is being used to send spam, they should ban the whole block to encourage the host not to allow spammers on their network. This tends to punish the innocent along with the guilty.

Spammer responses

• Find a host who will allow them to hop between IP addresses
• DDOS against the RBL
• Relay spam through zombies (generally home computers) on dynamic IP addresses

Advantages

• Can have a significant impact on the amount of spam received
• Runs at very little cost to the receiver of the email (no bandwidth spent receiving the email)

Disadvantages

• It can be hard to get off an RBL if you get on one
• The false positive rate can be quite high, depending on which RBL you choose
• If you have a false positive, you never know about it

Whitelisting

This works by storing a list of valid email addresses or IP addresses (generally just email addresses) that your server will receive emails from. In general this is not a terribly effective solution as it severly limits the list of people you can receive email from. This is typically to eliminate email from other testing criteria (eg to avoid running bayesian filters over it).

Spammer responses

• Joe job

Advantages

• Can have a significant impact on the amount of spam received
• Low requirements (badwidth, computation)

Disadvantages

• You can only receive email from email addresses/IP addresses on that list

Challenge - Response

This is really a variation on whitelisting for email addresses, with a dynamic white list. When someone who is not in your white list sends an email, an automatic email with a list goes back to them. Clicking on that link adds them to your whitelist.

Spammer responses

• Joe job

Advantages

• Can have a significant impact on the amount of spam received

Disadvantages

• Places a burden of work on the people sending ham emails
• Tends to work only if you have a small, known list of people who send you email

Greylisting

Greylisting is one of the more interesting ideas out there. Greylisting checks against an internal database to see if the combination of sender, recipient and sender IP address matches an IP address for an email that has been delivered. If there is a match, the email is received. If not, the receiving server sends a response to the sender to say that the server is unable to receive the email at the moment and to retry after a delay. This eliminates a proportion of spam by delivering mail only from MTAs that comply with the standards for email. The real power of greylisting comes when coupled with RBLs. If the email is part of a spam run, by the time the sending MTA resends the email, the IP address is likely to be in an RBL.

Spammer responses

• Run a complying MTA helps

Advantages

• Low bandwidth/CPU cost

Disadvantages

• Delays some emails from arriving immediately

SenderID and SPF

SenderID and SPF are two approaches to deal with one aspect of spam: Joe jobs. Both add records to the DNS records for the domain to list the IP addresses that can send emails for that domain. Of the two SenderID is technically a better tool, however Microsoft (the creator of SenderID) has patented parts of this. This makes it impossible for it to be implemented on most Open Source mail servers (postfix, qmail, sendmail, exim, etc), which make up a significant proportion of all mail servers. As a result we are unlikely to see SenderID implemented.

Spammer responses

• Run an MTA that supports this

Advantages

• Low bandwidth
• goes some way to deal with the Joe Job issue

Disadvantages

• Not supported by all MTAs, likely to drop some ham

Blue Frog

As far as I am aware there was only one implementation of this. The basic idea was to make a single http request to all links in all incoming emails. This would bring the sites hosting the products sold by the spam to their knees by the sheer volume of requests. Even if the servers could handle the load, the increased cost of bandwidth would make the spamming uneconomic. Please note that this is not a DDOS, as it is making just one request for each incoming email.

Spammer responses

• multiple DDOS

Advantages

• Hurts the spammers, adds costs to them in proportion to the emails they send

Disadvantages

• Not around any more :(. Unfortunately the DDOSes brought the service to an end.