Category: Technology

I’ve been kinda distracted throughout work for about a week now, because of the third Capture the Flag competition hosted by Stripe. The first CTF was on application security – you had a ‘locked down’ account on a computer, possibly with various programs available. And you had to break out of the account to read a file you weren’t meant to be able to access, containing the password for the next level. I wasn’t aware of that first competition.
The second one was on Web Application security – you were presented with web access to a web app, and the source code for the apps (each level in a different languages), and had to exploit various vulnerabilities in order to get the password for the next level. The web apps ranged from basic HTML form processing, to rudimentary AJAX twitter clones, to APIs for ordering pizza. The vulnerabilities ranged from basic  file upload validation to SHA-1 length extension attacks, Javascript injection all culminating in a level that involved using port numbers to dramatically reduce search space for a 12 character password. I completed that one and won a t-shirt.
The third one, the one that has just happened, was different. It was ‘themed’ around Distributed Systems, rather than security. You’d be given some sample code that you’d have to speed up, either by finding bottle necks in the code or by making the system distributed and fault tolerant. Spoilers will follow. Although the contest is now over, the level code (and associated test harness) is available here if you still want a go. I will note that it’s entirely possible to rewrite each level into a language you’re familiar with (I didn’t take that approach though, given that half the fun is not knowing the language).
So. To details.
I didn’t mange to finish it, although I made it to the last level, of which I sunk far more time into than was healthy – I’m fairly certain my tiredness at work for a couple of days was due to this level.
Level 0
Level 0. A basic Ruby program that reads in text, and if a word appears in a dictionary file it will enclose the word in angle brackets.
I didn’t know Ruby, but I had a good inkling of where to tackle this level, given how simple the program was. A quick google of Ruby Data Structures, and Ruby’s String split() method confirmed my idea. The original code did a string.split() on the dictionary and then repeatedly looked up each word against the Array that function returns. By transforming that array into Ruby’s notion of a Set, I could gain the speedboost from super-fast hash based checking.
I also modified the comparison to do an in place replacement as it saved the cost of duplicating the entire string. I’m unsure how much weight that had against the Array->Set change.
Level 1
A bash script that tries to mine the fictional currency of Gitcoin. Gitcoin is essentially like Bitcoin. You “mine” gitcoins by adding a valid commit to the repository. That commit must modify the ledger file to add one to your own total of gitcoins. A valid commit is one whose commit hash is lexicographically less than the value contained in difficulty – that is to say, if the difficulty contained 00001 your commit hash would have to start with 00000[0-F]. Because of how git works you have to find such a commit before anyone else mining against the same repository finds a valid commit.
There was one main thing in this level to fix. And that’s the call out to git that mock hashes the commit object to see if it’s valid. If it isn’t it alters the commit message text in some way, and then hashes again. This is slow. It’s slow because of a couple of reasons. Git likes to lock its repository files during operations, so you can’t do parallel searches for valid commits. But also because git objects have to have a very specific format, which git takes time to go and generate before returning the hash. The final thing is that each commit contains the hash of the parent commit as part of it, so naturally should another miner find a gitcoin before you, you have to start the search over again.
To achieve this, I moved the SHA1 testing over to Python. I formed the commit object that git creates manually – the header consisting of the term “commit ” and the length of the commit body, with a null byte. I left the body (which itself has to have a very specific format) as it was in the original script. I called pythons SHA1 library to do the work, which is a non-blocking operation, thus meaning I could set 8 separate processes going at once, each trying a unique set of commit messages. Upon success they then spat out a commit message into a file.
Annoyingly my solution then became quite clunky, with myself manually changing the filename to read in a copy of the original script that bypassed the searching. That pushed the correct commit. Ideally I’d have automated that into the first script, but it was enough to get me a valid commit pushed to Stripes git servers, thus meaning the next level was unlocked.
Incidentally this level had a bonus round, where instead of being against four stripe bots mining, you’d be competing against the other players who had completed the level. Needless to say, people very quickly started throwing GPU based SHA1 tools at it, and I was outclassed by a wide degree.
Level 2
Node.js, again I had no experience (although I do know javascript). You were given skeleton code that had to mitigate a DDoS attack. Your code would be placed in front of an under attack web service, and it had to ensure all legitimate requests got through, and strangely enough illegitimate requests to keep the servers busy, but not falling over. (You lost points in the scoring system for how long the target servers were idle.
In practise this was rather simple as the requests were easily differentiated – each legitimate IP would only make a few requests and relatively far apart. Simply keeping a log of the IPs seen and when they were last seen was enough to differentiate the legitimate mice from the illegitimate elephants. You also had to load balance between the two servers that were available – they would fall over if they had to process more than 4 requests at a time. You knew how long it each request would have before the backend servers timed the connection out, so by keeping a log of when each request was proxied, and to which server, you could check how many requests were likely to still be on the server.
Pretty simple.
Level 3
The penultimate level. Scala. I had great troubles with language on this one, I suspect partly because it’s close enough to Java that I get confused mentally when translating what I want to do into the scala syntax.
You were given four servers – a master one, and three slave servers that would never be contacted by the test harness. You were provided with a directory which you had to index all the files under. Then you had to respond to a barrage of search requests (for which you were also expected to return substring matches).
The default code was incredibly poor, so there were some immediate optimisations that were obvious. Firstly, the master server only ever sent search requests to the first of the slave nodes, which also had to index  and search the entire corpus. There’s two approaches now – split the corpus and send each search to all nodes, or split the searches but make each node index the entire corpus. I went with the former. I split the corpus based on root subdirectory number. So the slave0 would index when subDir%3 = 0. Any files directly under the root directory would have been indexed by all nodes.
The second obvious improvement was that the index was an object containing a list of files that the searcher needed to search. That object was serialised to disk, the searcher would read that in. Then for each query it would go off and load the file from disk before searching the file. My first change was to never serialise the object out, but keep it in memory. That didn’t make much of a difference. Then two options presented themselves. I could try constructing an Inverted Index – that would contain each trigram (as I had to handle substring searches)  and a list of the files and lines where that trigram was found. Or I could take the lazy option of reading all the files in at indexing time (you had 4 minutes until the search queries would start) and storing those directly in the in-memory index. I took that option. I transformed the index list into a HashMap of FilePath to Contents.  And that pretty much got me to pass. Somehow. I don’t feel like that was enough work myself, but that was more than made up for by the last level.
Level 4
I couldn’t crack this one. I tried for days. I think it was from Sunday through Wednesday, excepting some time out for the day job.
The language was Go. I know no Go. The challenge: A network of servers, each with a SQLite database. The network is unreliable with lag and jitter randomly added, and network links being broken for seconds at a time. Search queries will be directed to any of the nodes for 30 seconds. All answers they give as a network must be correct. You are disqualified instantly should you return an inconsistent answer. You gain points for each correct response. You lose points for every network byte of network traffic. Oh, and unlike in the other examples, the sample code they provided you with doesn’t pass the test harness – it gets disqualified for inconsistent output.
So. This level was about distributed consensus – how to get multiple nodes to agree on the order of operations given communication problems. I’m just thankful we didn’t also have to contend with malicious nodes trying to join or modify the traffic. If you could get traffic through it was unmodified.
The starter help text contained pointers to a Distributed Consensus Protocol called Raft. Vastly simplifying the intricacies: Nodes elect a leader. Only the leader can make writes to the log (in this case an SQLite Database). The leader will only commit a log once a majority of nodes have confirmed that they have written to the log themselves. If the leader goes missing, the remaining nodes will elect a new leader.
There’s a library already written for Go, Go-Raft. This seemed like a sure fire winner. Just drop in Raft right? Although dropping the library in was very easy it wasn’t that simple. Raft is a very chatty protocol requiring heartbeat signals, leader elections and in our case, request forwarding to the leader as followers do not have authority to commit to the log.
Beyond that though, the go-raft library had issues. It didn’t work with Unix Sockets (that the test harness required) out of the box (although Stripe had had a commit merged into Go-Rafts master branch that made fixing that extremely simple. It could fail to elect a leader. It also had a bug that seemed to bite a lot of people in IRC – I only saw it once, and I’m still not sure on what exactly the cause is – I suspect a missing/misplaced lock() that caused a situation with the log that is fatal for the raft consensus algorithm.
After battling with Unix sockets and getting an excellent passing score locally – at one point I got 150 points normalised, whilst you only needed 50 to pass, I pushed to remote. And it fell over horrendously. I ended up with a negative point score before normalisation. Needless to say that was demoralising. It turns out that reading the original Raft protocol paper, understanding it theoretically, and getting it to work with some easier test cases is very different to getting it to work in a much more hostile set of conditions.
My problems on this level were compounded by the infrastructure regularly falling over and needing the Stripe guys to give the servers a kick or 10.
But beyond that, I feel that there’s something I failed to grok. When my connections could get through it worked fine – SQL was always consistent, leaders were always elected, requests were forwarded properly (barring one case that I have since read about where the request is forwarded and executed successfully but the response is lost due to jitter).  And yet when running on remote I either suffered from End of File errors (i.e. socket closed), or requests timing out. Although I eventually managed to reproduce those issues locally by manually downloading the test case, it didn’t help me in diagnosing the problem – I regularly had a case where one node, in the entire 30 second test runs, never managed to join the consensus (which takes a grand total of one successful request to do). And I didn’t know what to do. I think that the most valuable thing this level taught me, beyond the theory of distributed systems, is how bad I am at fixing problems when there’s no errors that are directly caused by my code. As far as I could tell everything was fine – if I ran it manually without the test harness in the middle it all worked. But when I put the test harness in, it all fell over. My own logic tells me that therefore the problem must be with the test harness. Except I know people managed to pass the level with go-raft. I need to go and look at some solutions people have posted to see how they coped.
At the end of the day, however fun this was overall, the last level left a bad taste in my mouth – the infrastructure problems were pretty endemic especially nearer the end, and the difference between local and remote in the last level was absolutely disheartening. I can accept some difference, but a score that locally is three times higher than the threshold (after normalisation) shouldn’t get negative points on remote. I just wanted the T-Shirt!

In the vein of a previous post exploring why I chose to move my email over to Office 365, I shall today be exploring how I chose my new phone.
Or, more specifically the OS of the phone (given that hardware doesn’t interest me as a thing – one black fondleslab is much like another black fondle slab).
As that previous post indicated, I currently have a BackBerry. Not one of the new BBOS10 ones, but an older one (although it was new when I took my contract out).
The phone market today is radically different from the one where I first switched to BlackBerry (4 going on 5 years ago). BackBerry has essentially died a death ( in the consumer market anyway, we’ll see if their refocus back on enterprise, and the opening of BBM to other phone OSs makes a difference). Android has risen to become the dominant phone OS – although the device manufacturers haven’t quite got the hang of OTA updates and multi-year support (I’ll get to the issue of so-called bloat-ware in a minute). IPhone and it’s iOS has seen a more sedate rise, but has figured out OTA updates that cut the carrier out of the picture. Windows Phone has also emerged as a serious contender.
Between them, these 4 OSs have the overwhelming majority of the market – few people could name any other OS that is still going today. This post will take each in turn to weigh the advantages and disadvantages, for me according to my needs and desires. I make no claims that my answer is the one true answer, or that even my disadvantages won’t be someone else’s advantages. (although I am right, and everyone else is wrong).
BlackBerry
There’s no denying that BlackBerry has had a rocky road recently. Their latest OS 10 is a major shift from their previous direction. A major UI overhaul, coupled with keeping their excellent security features should stand them in good stead in this battle. But alas, I don’t want another BlackBerry – their troubles don’t speak well for being around much longer, or at the very least that consumers will not have the focus they once did. BBM is something I rarely use, and even if I did there’s no longer any need for a BlackBerry itself. Even email, the killer feature it handled exceedingly well, is no longer a differentiator – the competition has caught up, and BlackBerry hasn’t advanced. Their attempt to boost their App Store by making their OS ‘compatible’ with Android apps, to me speaks of a desperation. A last gasp as it were. Perhaps it will be enough, perhaps not. But I don’t want to be take the risk that I’ll be left with an unsupported brick a couple of years down the line (phones to me are at least a two year investment, if not more).
Windows Phone 8
A relatively recent contender, Windows Phone 8 is Microsofts latest attempt to break into the mobile market – a successor to the previous Windows Phone 7 and the Windows Mobile OS family. It inherits a lot of its look from Windows 8 and its Metro UI, and this certainly makes it the most distinctive of the OSs out there. Yet it hasn’t been a massive success, although it is showing steady growth. Perhaps it came too late to the market, or perhaps it hasn’t been marketed well – a common feature of Microsofts mobile attempts. One thing is certain though – app developers haven’t gone crazy for it. Despite the fact that I only use a core set of apps on my phone regularly (mostly social media), I do like to try out apps, and part of me wonders if it’s due in large part to the fact that BlackBerry’s app selection is abysmal.
iOS 7 on iPhone
I already have many Apple devices. I use a Macbook Pro at home, I have an iPod Touch which is my media center, and I have an iPad which sees infrequent use. I have a large collection of apps on my iPod, although again I only have a core set that I actually use. So surely an iPhone is a natural next step? Well, maybe not. iPhones are expensive (I know, that’s hardware – but unlike the other OSs, Device and OS are tied together here). I already have an iPod Touch for all my Appley needs. I know of no-one who uses iMessage or FaceTime – so those have no appeal. My apps are already on my iPod Touch, and I don’t hate the wifi-only nature of it. There’s also Apples iCloud, which is very much a walled garden as far as syncing services go. I use it as minimally as I can for my needs right now (mostly to save connecting via cable to transfer photos).
Android
Oh Android. Google’s attempt at a mobile OS. Phenomenally successful. Open Source, except for when it’s not. Android. It came on to the scene with a terrible UI at the time, although the UI has improved dramatically with recent revisions. But then, with Android, the UI is kind of moot. It’s open source (except when it isn’t), people have written entirely separate launchers and themes – see many of the carrier/manufacturer branded versions for examples. In fact, this really makes it very hard to talk about Android with any meaningful detail. Google’s Android is very different from the Open Source Android – the Keyboard with the cool swipey-pathy-typey-thing? Closed source. Googles Mail app? Closed source. It’s well documented that Google has been closing down Android slowly but surely. And although you have the possibility of side-loading apps, very very few are actually distributed like this. They almost all go through Google Play Store. It seems that Open Source is a flag google wave for community support, to blind the community to just how hard it actually would be to create a successful Android fork – look at what Amazon has to go through to clone the APIs provided by the closed source Google Play Services. CyanogenMod also have to dance around the redistribution of the closed APIs that many apps assume are present, by backing up the original Google Apps, and then reloading it after their version is flashed. Also, how meaningful waving the Open Source flag is when the core platform APIs of the project are developed in private is…….. yeah.
I make no secret that I don’t trust Google these days. You are an advertising target to them. Everything they do that is intended for consumers will eventually feed back into their advertising algorithms. This is why it will surprise you that I went with Android as my next phone OS. I’m not sure yet, how I’ll remove or limit Googles tendrils on the device. Running stock AOSP? Possibly if I can get my social media apps to work without the Google Play Services. Using a separate account for Play Store things? Possibly. I’ll most certainly be limiting apps permissions as much as possible. I was surprised to learn that Android only recently got the ability to limit GPS access on a per-app basis – iOS has had Location Services control for ages. Perhaps I’ll put CyanogenMod on it, although frustratingly I can’t find a full description on their site of what changes they actually make to AOSP. I’ll certainly disable Google Now, and its always listening “Ok Google”. I’d better buckle up, because this is going to be an interesting ride. Especially as I find apps I just want to try if only for 5 minutes.

So, there’s been…. a lot of panic in the Tor community over the last 24 hours. Let’s have a look at some facts shall we?
Firstly, it would be good if you knew some basics of Tor – I have a previous article on it here. Secondly, forgive the number of Reddit Comments I’ve linked to – but given the lack of mass media coverage of this news, there’s not much choice)
News broke that the FBI had issued an arrest warrant and extradition request to Ireland for Eric Marques. The article frames him as a large distributor of Child Abuse Images. Whether that is accurate or not remains to be seen in court, but one thing that is (now) known is that he was the man behind “Freedom Hosting” which provided hosting for Tor Hidden Sites. A number of those sites apparently hosted Child Abuse Images or videos. It’s not yet known if he had any connection with any of those sites beyond being their hosting provider.
One immediate question that presents itself is how did they find out that this guy was operating the Freedom Hosting site? I haven’t seen any evidence on how this happened. It’s possible that they used a server exploit to find out the machines real IP address. Or that they tracked him down via other means (financial records etc), and then happened to find out he was behind it. Incidentally, the only evidence that the Tor community has that he ran it was the timing of all these events.
So, all the sites hosted by Freedom Hosting disappeared from the Tor network. Then, a few days later they showed up again. But this time, some (but not necessarily all) the sites hosted included an extra iframe  that ran some javascript code (Link is to a pastebin, so is safe to click). Needless to say this javascript code is an attempt to break anonymity.
Now, a small amount of background. Tor (for end users) is mostly run through the Tor Browser Bundle these days. This combines Tor with a patched version of Firefox – to fix some anonymity leaks, as well as some Firefox extensions such as HTTPSEverywhere, and NoScript. NoScript is a Firefox extension that prevents Javascript from running according to the users preferences (block all, whitelist domains, blacklist domains, block none). Great, so the Javascript wouldn’t run? Well…. no. Tor Browser Bundle ships with NoScript in the “run all scripts” mode. Tor have had an FAQ about this setting up for a while. The short answer is that because Tor tries to look like a normal machine – always reporting a Windows NT Kernel (even on other OSs) for example, that disabling JS would leave you in a minority, as well as making it harder to actually use the normal javascript-reliant internet. Needless to say, Tor are reevaluating this tradeoff. This is especially true as their patches to Firefox should, in theory, make it harder for Javascript to break out and find the users normal IP.
So, this script can run. What does it do? Well it specifically targets Firefox 17 on Windows. Firefox 17 is the Extended Support Release of Firefox, which is what the Tor Browser Bundle is based on. Claims that this is a 0-day attack have been abound, but further examination has revealed that in fact, it had already been patched in Firefox 17.0.7 – which had been packaged into a Tor Browser Bundle at the end of June/early July. When you put this together it means that the script only affects users of old Tor Browser Bundles on Windows. The script appears to use the vulnerability above to try and send your real IP to another server. It also tries to set a cookie, presumably to track you as you browse the internet and onion land.
Notably TorMail, (a service which provides public email facilities over Tor), was also apparently hosted on Freedom Hosting, so far more than just people accessing Child Abuse Images are potentially affected. Anyone who wanted a truly anonymous email account has been affected. This makes it likely (although not guaranteed) that the FBI now have access to every e-mail stored on that server.
Freedom Hosting, whilst not the only Tor Hosting Service, was certainly one of the largest and well known. And TorMail was unique in its service. What this will mean for whistleblowers and others who used TorMail remains to be seen.

I’ve recently been able to take advantage of UltraViolet ‘digital copies’ of movies, and the experience has left me with some thoughts. I promise that this won’t turn into an Anti-DRM rant, beyond a brief mention of the rights you are guaranteed.
Firstly, for those who don’t know what UltraViolet is, a brief explanation. UltraViolet is essentially a service, backed by 5 of the ‘Big 6’ movie studios, that provides a ‘digital locker’ of sorts for digital copies included with select movie DVD/BluRay discs. You buy the DVD or BluRay in the shop, and included in the case is an ‘UltraViolet’ redemption code. After following the instructions you then have a digital copy of the movie in your UltraViolet account that you can then stream or download onto your UV compatible players.  There’s also a feature of adding up to five additional accounts under your own to share UV library access.
At least, that’s the ‘non-technical’ version. What the service really is, is a collection of license files. That’s it. The Ultraviolet service itself doesn’t store any movie files, it just stores some DRM information stating that you have the license for this film. The actual files, streaming and downloading services are provided by other ‘UltraViolet Retailers’ – so, for instance a Warner Brothers movie will likely use the Flixster service – both being owned by the same parent company. The ‘UltraViolet’ website ‘stream’ and ‘download’ links merely point you to this other service. Which is an entirely separate account. UV mandates a choice of DRM schemes, and mandates a few select rights (streaming for free for one year, unless geographic restrictions come in to play), all other options are couched in “may” language, and thus might not be applicable at any point.
Because of this… decentralised…. nature, the core of ultraviolet is essentially nothing more than a thin veneer on-top of numerous different digital lockers. And thus is pretty useless. There may be progress in the future – UltraViolet appears to have plans to create ‘branded’ UltraViolet players that let you stream (or play downloaded files) from any of the UV retailers. This unification could actually provide some value.
Now, let’s talk about this in practise.
My first experience was when I bought Les Misérables on DVD. For this film they had also allowed an additional iTunes redemption (separate from the UV redemption – as UV is incompatible with iTunes[0]). iTunes redemption was simple – sign in/up to the iTunes store. Put the code in the redemption box, and done. The UV experience first required me to sign up for a Flixster account, then allow that to create a separate Ultraviolet account, put the code in the box, and it showed up in my UV library, and in my library on the Flixster site. As I only bought the DVD version, I was not surprised when I only received a Standard Definition digital copy. This however is where the cracks start to show. On the UltraViolet site the ‘download’ box is greyed out – implying that I only have streaming rights. The ‘watch’ link on the UV site simply takes me to my Flixster library. It doesn’t start the streaming of the movie – instead I now need to hunt again through my Flixster library to find the actual movie. On the Flixster site however it transpires that I can download the movie – into the Flixster desktop app. It’s this inconsistency of information, and the lack of one click integration that, to me, is UltraViolets biggest weakness.
My second experience was redeeming my ‘Harry Potter Wizards Collection’. This is essentially a giant box containing all 8 movies on DVD, BluRay, and a UV copy of all the movies, along with many special features and exclusives (No iTunes digital copy code however). The UV retailer was again Flixster, although this time I only needed to sign into Flixster, as they had ‘linked’ my UV and Flixster account together. Thus my previous notes about this particular retailer carry through. This time, however, on the UV site it shows that I have the standard and high definition version of all the films. However when I go to the Flixster site, I can find no way to actually obtain, through stream or download, a High Definition version of the films. This is a major problem in my view – and again, highlights how the lack of unification and integration is too apparent. When they’re trying to compete with iTunes for digital distribution, unification is one thing they need to get spot on. To be told in one place that you have the High Definition version, but not be able to get it is frustrating.
If UltraViolet feels like one thing, it really does feel like it was designed by a committee. None of whom wanted to let go of their vested interestes,  each of whom has a different idea about what should be allowed, leading to this incredibly weak platform. If their UV Player idea ever takes off, then it would help. But fundamentally, the idea of a system that does nothing but say “John Smith acquired this movie on this date from this UV Retailer” is worthless on the technical side. To the end user it’s a glorified catalogue that doesn’t even provide one click access.
A final note – obviously I haven’t tried any of the other UltraViolet retailers, but I don’t have high hopes of them being any better either.
[0] – UltraViolet movies cannot be played in iTunes, nor loaded onto native iOS devices. However, UV Retailers (such as Flixster) do have iOS apps and native platform apps that have to separately download/stream the movie each time.

Everyone has the right to freedom
of opinion and expression; this right
includes freedom to hold opinions
without interference and to seek,
receive and impart information and
ideas through any media and
regardless of frontiers.

– Article 19, The Universal Declaration of Human Rights

 
I make no secret of the fact that I use Tor, and that I run a Tor Relay. Admittedly  I don’t use Tor as often as I perhaps should – but a lot of my browsing is on websites where I have an account that’s already associated with ‘real-world’ me, thus negating the purpose of Tor. Given the recent headlines, I figured now would be a good time to explain what Tor is, and why I encourage its use.

What is Tor?

Tor is software developed by the Tor Project that aims to ensure that your ISP and middle-men cannot correlate who you are with what you’re doing on the internet. The idea is that you download the Tor Browser Bundle – a one-click package of everything you need. Then when you browse the internet using the provided browser (a modified Firefox) you are routed through 3 volunteer relays in a way that guarantees forward secrecy, before the last node actually sends your request to the website in question. The way the encryption is set up means that when using Tor the ISPs, and anyone listening before your data reaches the first Tor node knows only that you’re running Tor, but not what you’re doing. The 3rd ‘exit’ node, and anyone listening to the connection between them and the destination website can only see your data, but not the original source. As far as the website is concerned your IP is that of the exit node. When you combine this with HTTPS connections secured with SSL even the exit node can only see the site you’re visiting, without seeing any of the data being passed back and forth. The EFF has a nice diagram summarising this.
Thanks to this it provides a way to secretly access websites that you may not be able to. In Turkey for example, it can bypass their web firewall so users can read news about what is really happening. Tor’s usage skyrocketed during the height of the recent Egyptian revolution.
A second feature of Tor are its ‘Hidden Sites’ these are websites who accept connections directly from Tor without having traffic go back over the ‘clear net’. This way there is no ‘exit’ node to spy on your data or site. Your connection is fully encrypted.  There are Wikileaks mirrors for those wishing to view that data. There’s an e-mail service so you can send and receive emails (even interacting with clear web email addresses) entirely anonymously. The New Yorker has an anonymous document submission/communication platform as a Hidden Service (the second link won’t work unless you are using Tor).

What’s a Relay?

A Tor Relay is a computer with the Tor Software installed that has volunteered to be one of the middle-men in other peoples Tor connections. They provide the bandwidth that can make the Tor experience faster and more stable. An ‘Exit Node’ is a special kind of Tor Relay that has additionally volunteered to be the last Tor node before a connection jumps out onto the clear net. By running a normal relay I can help ensure that whistleblowers and dissidents can access the information they need to do their job.

Any downsides?

Like anything, it’s not perfect. There are known attack models – if all three of the relays you route through are run (or have traffic logged) by the same organisation, they could in theory perform timing analysis to work out which data stream is yours. There’s also the fact that it is significantly slower – streaming data is out of the question (even more so because Tor is TCP and doesn’t innately support UDP). And naturally like all privacy preserving tools, it can be used by the bad guys as well – Tor semi-often hits the headlines because of the Silk Road – a drug marketplace that operates as a Tor Hidden Service. But humanity is mostly good. Some people can do terrible things, but in the grand scheme of things I believe that the good it enables, and that humanity uses it for, greatly outweighs the possible negatives.

So who the heck is funding this?

One last point. The Tor Project makes no secret of the fact that they were originally founded and are funded by the US Navy. But as you can see, there is now a wide diversity of funding coming in. And if that isn’t enough, Tor’s Projects are entirely Open Source (including the core Tor code) – you can download the code, submit patches (please do that!), and check there are no back doors.

Did I see mention of a free T-Shirt?!

Yes. Yes you did. I have my free t-shirt. Get yours for running a 500KB/s relay for two months. Or a 100 KB/s exit that allows Port 80 (HTTP) traffic. (I do the former).
—–
Privacy is important. Especially for those whose governments are actively stopping their own people from being well-informed.  You can make a difference. ^_^
 
Update: I finally found a link I was looking for, but couldn’t find. I’ve added it to the Hidden Sites discussion.

So, as you might have gathered from the title, I’ve switched away from Google for my E-Mail, along with Calendar and contacts, I figure a post on how I went about it is in order.
Firstly, my setup, and thus how I picked Office365 to replace Google Apps for Domains.
I have a Mac, and use Mail.app, Contacts.app and Calendar.app on there. I have an iPod Touch and an iPad, where again I use the default apps. My Phone is a Blackberry (OS 7, not 10 – this becomes important later). When using Google to house Email, Contacts and Calendar everything worked, and I linked my Facebook Account on the Blackberry to my Contacts application so my contacts had display pictures and additional contact information.
I only require one account, with a custom domain name. I may add further domain names in future.
The vast majority of my contacts are simply a name and email address. Others are a name and phone number. A scant few have both email and phone – unless they are linked up with Facebook.

Evaluation of Options

I wanted, as I stated before, a hosted solution. A friend recommended Intermedia to me, however they now require a minimum of 3 accounts which makes the initial cost of an account for me at about £18 per month. I also considered running my own contacts and calendar software, and going with a less full featured email service such as FastMail. Software I looked at included OwnCloud and Baikal. However none of the software for Calendar seemed particularly well designed – the web UIs were often lacking or non existant.
Looking, as I mentioned into Microsofts’s own solution. At £5.60/month I get unlimited storage, and a full email, calendar, contacts house. An additional bonus – important for me, is that it includes complimentary access to Blackberry’s Business Cloud Service. BCS is kind of a slightly scaled down version of Blackberry Enterprise Server, that Blackberry host themselves, and is directly integrated in to Office365 if you choose to enable it. This is vital as adding the Office365 account to your Blackberry Internet Service only synchs the Email, and not the calendar or contacts. Both of which are pretty essential for my phone to achieve its purpose.

The Migration

Having selected Office 365’s Hosted Exchange plan, I signed up and paid my first £5.60. The signup was quick and smooth, as was the account creation. When you sign up you initially get to choose a subdomain underneath onmicrosoft.com for your account. You cannot remove or change this subdomain, but it’s free to add your own domains to the solution, and assign users to the correct domain name. Once the account was set up I setup access to the default subdomain account on my devices, and sent an email to check everything could work. This necessitated enabling, and setting up my Blackberry Cloud Services account.

Blackberry Cloud Services

This is the only negative experience I’ve had. This was the first time I got an error – when it tried to load their administration panel. It fixed itself on a refresh, but still. In addition, unlike Office365 it is not a sleek UI. It is very much a business/corporate designed UI. It reminds me very strongly in fact of this comic image. Every navigation element leads to a giant search form with many options. Naturally, in my use case, with only one account and one Blackberry, I’d prefer just to see my user details straight away. That said, my use case is obviously not the primary target for this tool.
Once I’d searched for my user and selected myself from the search result list, I checked through the options given for configuring a Users Profile. On a Blackberry, this profile can control every little thing that the device and users can do. It allows specifications of device passwords, remote wiping, separation of work and personal apps (to the extent of disabling copy-paste between the two domains). Naturally, this web tool only provides a fraction of the power, and the default options were fine in my case. Thus I used the option to send an email with an activation password to my email. On the Blackberry I had to download the ‘Enterprise Activation’ app, and use the password provided to associate my blackberry.
All the testing emails worked fine. So I set about doing the actual migration

Doing the Migration

The first thing was adding my domain to the Office365 account. This was incredibly simple, and contained helpful instructions on updating the DNS to verify ownership, and enable specific services, for a variety of domain name service providers. Having verified my domain, and activated it for my Exchange account I reassigned my user to use my own domain instead of the onmicrosoft.com subdomain. That done I removed Googles Records from my DNS. I then set up the email migration. Again, Office365 had an excellent Wizard that catered for a variety of Exchange-Exchange migration situations, and a generic IMAP importer for other situations. I went with the IMAP route naturally. After providing the domain and IMAP server, I uploaded a CSV containing the details of the accounts to migrate (one in my case), and set the import going. With 27,000 odd messages it took a few hours for my account to fill up – so many messages due to the way GMail handles its tags system over IMAP. Each tag is its own folder, and messages are duplicated across the ‘folders’.
I then used my own Google Account to export my Contacts into a .csv file which I then imported directly into my own account, again without any issue.
The Calendar app however  as far as I can see has no built in import function. Trying to use Calendar.app to export my Google Calendar, and reimport it into my Exchange Calendar with its own import function caused an error. As I don’t actually have any future appointments scheduled at the moment, this is bearable. My history is currently only viewable on my Mac as I imported it into a local Calendar. For those with future appointments though that could well be a stumbling block, or possibly a deal breaker). You may be able to use Google Calendar to ‘share’ the calendar to the Exchange one, but I didn’t investigate that option as the loss is minor.
Having imported everything I updated all my sync settings to point to the new domain, and everything was pulled down fine. I ended up with Duplicate Contacts in some instances, but Merging them when they showed up on a device was sufficient. My Blackberry I had to remove the syncs entirely as it was still trying to sync to my old account as well. After wiping out all the contacts locally and re-enabling sync to my new account everything seems to work.
I went to re-enable Facebook-Blackberry integration, only to discover that this option is disabled by Blackberry Business Cloud Service. It’s configurable in the full BES package, but it is one of the things they removed in this not-quite-BES-in-the-cloud version. For now I have simply enabled Facebook integration on my Mac and iPod/iPad. The information added by these isn’t synched up into Office365, so my Phone contacts aren’t Facebook integrated at the moment. Microsoft however do have US only facebook integration directly in Office 365. Hopefully that will come through soon. Either way, Facebook contact integration on the phone is only a nice to have and didn’t really benefit me aside from the display pictures.
I did however enable LinkedIn integration in Office365, which pulled in additional contacts into my contact list.

Summary

Everything went better than expected. Office365’s Exchange component is fast, sleek and very nice to use. The only things that went wrong are a lack of Calendar import; and the Blackberry which had by far the most issues in all aspects of the migration. Something that can be laid firmly at Blackberry’s feet. That said, BBOS10 doesn’t have BES, but instead integrates with Email, Calendar et al over standard protocols (hopefully reducing those issues?).