Dear USA, my data has left your building.

NSA, GCHQ, Prism, FISA, Project Bullrun, Sigint.

After Edward Snowden, former CIA and NSA employee, started revealing how massively, intensely and easily we are all being spied upon by the intelligence agencies of various governments, the terms above have suddenly been spending a great deal more time in the media.

It turns out that government agencies are allowed to extract, at a whim, your and my data from service providers, such as Google, Microsoft and Yahoo. There is no real legal process (unless you can call a secret judge in a secret court giving a secret order a real legal process), especially if you’re not a US citizen, and the providers that have been forced to give up your data in this way are not allowed to notify you about your digital self being violated. So even if they say that you shouldn’t worry, you can never be entirely sure.

Furthermore, it has also been revealed that the NSA has for years being acquiring encryption keys via legal (secretly forcing companies to give them the keys) and extra-legal (simply hacking into company servers) means. Even worse, they have for years been deliberately introducing security weaknesses into software products and encryption software in order to be able to crack open your data even more easily.

You can read more about this state of affairs in The Guardian’s NSA files. The Guardian has been doing a sterling job of analysing and bringing to light the depths to which our governments have sunk. There’s a whole lot of information, and most of it is quite upsetting.

For me the final straw was when secure email service lavabit voluntarily shut itself down, when faced with the prospect of being forced to leak user information to the US government without being allowed to tell anyone. The message on the site is quite chilling, and concludes with the following:

This experience has taught me one very important lesson: without congressional action or a strong judicial precedent, I would _strongly_ recommend against anyone trusting their private data to a company with physical ties to the United States.

At this point, I was a super happy and pretty heavy user of a number of US-based services, including GMail (all my email, about 40000 conversations consisting of 60000 mails, that’s excluding my work email which I also hosted on GMail), Google+ Photos (all my photos, about 21000 of ’em), Google Drive, Dropbox (50G of data spread out over 120000 files). In all cases, I still consider these to be best of class services. In putting my money where my mouth is, I was paying both Google and Dropbox for extra storage.

I also had no problem with Google filtering through my email to show me targeted advertising. This is the deal I had with them. I also had no problem with the possibility of someone getting my data after due legal process. However, the idea that some NSA or other government agency flunky could quite easily stick their grubby paws into my data, and that I would never know about this, was too much.

There’s probably nothing much of interest in my data. However, it has become a matter of principle; Privacy is a basic human right. Here’s an old essay by Bruce Schneier if you need to read more about why privacy is so important.

In short: It was time to extricate all of my lovely data from probably well-meaning US companies, thanks to the ridiculously powerful and secretive NSA, and thanks to all of its shadowy counterparts around the world.

Here’s how I did it:

• Considered building another low-cost Linux server, or even a Raspberry Pi. Decided against this due to time required for configuration and acquired a Synology DS213j NAS, which is at this moment standing on the desk about 1 metre to my left. My recommendation: Just get this, you won’t be sorry.
• Downloaded 60000 emails to Synology using Thunderbird mail client. Deleted everything from GMail. Google engineers assure me that after a few months, data will really be gone.
• My webhoster (WebFaction) receives mail for all my domains. My Synology retrieves mail every 5 minutes via POP (you can set this up via Roundcube on the Synology) and deletes it from WebFaction.
• Outgoing mail is relayed by the Synology via the WebFaction SMTP server. I don’t have to worry too much about blacklisting and whatnot, my hoster does this.
• I’m back to interacting with my mail using Thunderbird and IMAP SSL. The loss of GMail conversation view was initially really REALLY painful. People have forgotten the ancient art of quoting. However, I’ve configured Thunderbird to archive all mail to year-stamped archive folders, and to put my sent mail there. Poor-man’s Conversation View! (the conversations plugin is wonky. it’s shocking how much the availability of GMail, which works really well, has stunted the development of alternative email clients) Importantly, I am now able to use OpenPGP again for the strong encryption and cryptographic signing of my emails.
• On my Android telephone (whoops…) I am using the Kaiten IMAP client.
• All the data I had in Dropbox is now being synced between the Synology, two laptops and a workstation using BitTorrent Sync. This peer-to-peer syncing system is still a little rough around the edges, but falls squarely in the category of “Best Things Since Sliced Bread”, and it’s FAST. CloudStation, Synology’s dropbox-inspired solution, was just far too slow on my Synology model.
• My photos (21000 of them) have been downloaded from Google+ Photos (thank you Google Takeout) and are now being served from the Synology using PhotoStation.
• I make incremental backups of everything to an encrypted external USB drive, using dirvish. I will probably add an extra external drive to the mix and try to keep that off site.

It’s been an interesting process moving my stuff out, and getting used to these alternative systems is sometimes slightly uncomfortable, but I am quite happy with the end result. I hope that more people will take this step, and I really hope that more and easier-to-use alternatives for secure email (such as mailpile) and for ubiquitous private data will become available.

My submission of this post spent some time on the Hacker News front page, and from there was picked up by reddit as well. This brought many comments, a number of which were positive and thoughtful, and a number of which that were far less so. It’s amazing how anonymity and comment sections can bring out the worst in people. (if you have to know, the Hacker News community is generally MILES more polite than reddit)

In any case, I wanted to clarify an issue or two: After moving my data away from GMail and Dropbox, I am not under any impression that my data is now secure. I can still be hacked. My hardware and software could be full of backdoors. My email will still be read as it jumps from server to server, probably ending up in someone else’s GMail. :) However, if more people were to move their data out to their own premises, it becomes more complicated and costly for government agencies to monitor us all. At the moment, the NSA cuts deals with a few large email and other cloud service providers, and with that they’re able to monitor large swathes of users. However, if more of those users were to move away, many more deals have to be cut and servers hacked, costing more time and more money. Add to that increased used of OpenPGP (which I do use, and mention in my post), and it becomes even more difficult. I know that I’m just a drop in a bucket, but hey, at least I am a drop in a bucket!

My goal with posting this was to show that it’s relatively easy to move much of your data away. I have the feeling that many of the most impolite anonymous commenters still store their data with cloud providers, and would really prefer to believe that there are no worthwhile alternatives, hence all the ad hominem attacks.

Fortunately, each polite and humane comment makes up for a whole pile of bad ones. :)

Google Drive: Not reliable yet, but potential.

I’ve been a Dropbox Pro (50G) user for more than two years now, and in this time it has never let me down, not even by a little bit. Still, when Google announced its new Google Drive syncing service, I had to take it for a spin.

For those of you with short attention spans, my conclusion is: Google Drive has great promise due to its price-point, Google’s great infrastructure and the integration with Google Docs, but you shouldn’t yet trust this service with your critical files.

To summarise: Google Drive is Google’s answer to Dropbox (and 50 other inferior syncing services). You install a small app on your Windows or Mac (no Linux yet, although it has been promised), and then it’ll keep a folder of your choosing in sync with Google Drive in the cloud. You can access your files via the website drive.google.com (google docs, but slightly updated), any computer with the Google Drive software, or via the Google Drive mobile apps.  You can also share files through authorizing the relevant google accounts, or via URL. Google Drive has a number of built-in viewers, meaning that users will not have to install PowerPoint to view your PowerPoint presentation for example.

Things start to deviate from Dropbox when we look at the storage plans and prices the big G is offering:

You get 5G for free. For a measly $2.49 per month, you get 25G of storage, and 30G of GMail storage as a bonus, and for$5 per month you get 100G! Compare this with the $10 / month Dropbox wants for 50G. This, together with the fact that you could go up to 16 TERABYTES if you would want to, makes you at least think for a bit. I installed the client on this Windows laptop. For you screenshot-freaks, here’s the context menu for the systray icon: Note that because I pay a measly$5 / year for 25G of extra GMail space, I’ve been grandfathered into 25G of Google Drive space. Heh, I also only just learnt the term grandfathered. It means I could get this because of my previous price plan that doesn’t exist anymore.

Here’s the preferences dialog, nothing special really, unless you have a screenshot fetish:

For me also an important functionality: You can easily recover deleted files. If you delete a file on your client computer, it gets synced to the trash folder on the google drive website, from where undeletion is an easy click on the “Recover” button away. Under the File | Manage Revisions you can retrieve file versions up to 30 days ago, or 100 revisions, whichever comes earlier.

Another important difference is that Google Drive, as far as I could find out, does not do something similar to Dropbox’s LAN sync, a pretty cool function that will grab files from the computers on the local LAN if they’re available, instead of from the cloud.

So I set out to do some tests. Before I could really get started, I ran into the first problems. I created a text file in my Google Drive folder with Vim (yes, I use Vim. deal.), as I wanted to test the file revisions. As you know, when Vim saves a file, it first writes to a temporary file, then deletes the original file and finally renames the new file to the original file. This confused Google Drive to no end. For each save, Google Drive created a new file with exactly the same name in the web interface, whilst on the client side, there was only one file.

I then proceeded to delete the text file on the client, leaving me with the following situation, even after Google Drive was done syncing:

As you can see, on the server is my text file, on the client nothing. I expect of a syncing solution to actually, uhm, synchronise my files. I did notice a sync error message in the systray context menu. After clicking, I got this dialog:

Yes, thank you Google Drive, you have an unknown issue. That’s just great.

So, in spite of the really attractive offering, this type of wonkiness (multiple files due to stupid create-new-rename saves, sync errors soon after), even after a few minutes of playing around, does not instil confidence or trust. If there’s one thing a good sync service should do, it’s instil confidence and trust. Dropbox has never failed me, and I’ve thrown some pretty strange things at it. Until Google Drive is able to do the same, I’ll continue coughing up 10 bucks a month for Dropbox.

The Monthly [Weekly Head Voices #50]

HEY!

I’m still here, and it seems I really have to catch up on my backlog of WHVs, all the more as I was starting to notice the beginnings of BPP (Backlogged Posting Paralysis, of course). So I’ve spent a few minutes gathering a selection of life snippets of the past six weeks (week 21 through week 26) and will now proceed blasting them out this old Web 1.0 exhaust. I wasn’t completely idle blog-wise, however. I did write a post about my EuroVis 2011 and my Schloss Dagstuhl SciVis seminar adventures.

Before the blasting commences, I would like to present some relaxing visual input brought to you via my cell phone camera, which at the time of capturing found itself in my hand, itself being inside the chapel in Herberg op Hodenpijl, a short westward cycle from my house:

The picturesque surroundings are home to the chapel, which hosted an art exhibition at that point, and an organic restaurant and grocer. Most (all?) of the produce comes from a small farm across the road that you can also visit. The goats are really friendly. You could do worse than popping by on a sunny day.

The rest of this post has been categorised, with nice headings, so that you can skim through it even faster.

Health and well-being

• In a recent cooking insert on the television, two chefs prepared Loup farci en croûte, or sea bass filled with julienne vegetables in a pastry of a thousand layers. Take a look at the video clip: The chefs put an amazing amount of effort into preparing this visually beautiful and apparently delectable dish.
• My TNR and since recently also business partner, who can often be found hurtling down mountains on various and high-speed forms of personal transportation, and when he’s not is involved in a number of other extreme sports activities, managed to break two fingers on his right hand cycling over the flat and otherwise uneventful piece of earth between the computer science and physics buildings on our campus. Go figure.
• In a recent study with 48000 (yes, that’s fourty eight thousand) men followed over a period of 22 years, a strong correlation was found between drinking six cups of coffee per day and a lowered risk of prostate cancer. Also men (but can you still call them men?) drinking fewer than 6 cups of coffee per day had a lower risk. The study did correct for other lifestyle factors. The linked summary also mentions other studies in which coffee drinking has been associated with lower risk of Parkinson disease, type 2 diabetes and liver cancer. I guess I can worry less about this addiction than, euhm, the other ones.

Nerd News

• Dropbox has recently activated functionality they call shareable links, meaning that you can request a unique http://db.tt/some_code link for any file or directory anywhere in your dropbox and share it with anyone else (also non-dropbox users), who is then able to download said file or directory given the link. There’s a page on the website where you can manage all of your shared links, for example deactivating ones you don’t want people to access anymore. Read the help on shareable links for more information.
• As you might know, I keep a lab journal documenting in some detail my daily work activities. I also maintain a personal journal, mostly for dumping stuff that might be interesting for this blog. I used to do all of this in Google Docs, but for the past few weeks I’ve experimenting using TiddlyWiki (this is a single file JavaScript-heavy wiki implementation) on my Dropbox for all my journaling and personal knowledge base needs. It’s been going swimmingly for at least two reasons: 1) It’s available also when I’m offline. 2) The idea of being able to break out into a new wiki page (called a tiddler) at the drop of a hat takes some getting used to, but fits the non-linear nature of my journal and personal knowledge base entries much better than the mostly linear google docs.
• I’ve settled on using the free tonido personal cloud software to make all the files on my home server available via the internets. This means I can get to all of my music, photos are whatever no matter where I am. Pretty neat! (I have too much to fit on my 50G dropbox account. There’s also the PogoPlug software, but the free version has its limitations.)
• Just a few days before Google+ (Google’s new social networking religion, in case you’ve been sleeping for the past week) hit the internets, I installed the Google +1 button on this blog, see right at the bottom of every post. So now you know what to do with every post: Click my +1 button, then click my facebook like button, then leave me some snarky comment right here. Easy as 123, and I’m a happy camper.

Comedy

Whilst flipping through channels one Saturday night,  I flipped right into the stand-up comedian Demetri Martin. I made a note of it in my journal, and now here we are. Watch him present his research findings on his large pad, with his pointer:

That’s it for now dear readers. I really do hope to be back soon, and I hope to do so with a slightly more focused contribution. See you on Google+!

UPDATE 2011-07-23 note-taking strategy

To you I might appear fickle, but I guarantee that it’s just hyperactivity. I’ve since adjusted my note-taking strategy again. As you will have seen in the comments, Pieter Kitslaar subtly influenced me to go searching for a note-taking solution that syncs between phone and everywhere else. Fortunately, I ran into the SimpleNote universe and I’m now officially in note-taking heaven. I have Flick Note on my Android, the SyncPad extension on Chrome (for sometimes) and ResophNotes on Windows and on Linux with Wine. All apps work exactly as I would expect a super-fast and efficient note-taking app to work, with real-time incremental searching, hotkeys everywhere, and best of all, offline use and transparent syncing. ResophNotes is especially cool, definitely give it a shot.

So I’m using this SimpleNote setup for all my personal knowledge base and general note-taking needs. For detailed work and lab journaling, I’m now using linear OpenDocument files on my Dropbox, which solves the offline problem I had with Google Docs, which I do still adore for collaborative work.

Lemme know in the comments what you think!

Weekly Head Voices #19: The time-traveller’s BBQ.

This one is dedicated to the memory of DB.

In this slightly introspective 19th edition of the Weekly Head Voices, looking in all directions in time simultaneously, but slightly ever so slightly more in the direction of week 12 of 2010, I do the usual week time break-down, win a fight with my ISP, go all out on Dropbox and Chrome, and smash through my yearly internet shopping quota, before arriving at the more extensive than usual backyard philosophy section, which today will dwell on the very much related themes of time-travel, perspective, stress and BBQs.

During the past week, I checked off 22 GTD tasks, spread over 10 projects.  Besides these, I had the usual 2 hours of lecture preparation, 3 hours of lecturing and 17 hours of scheduled meetings, including an absolute power networking event where I definitely was the smallest fish in the bowl (lovely that!).

If you are not of the nerdy persuasion, you might just want to skip the next few paragraphs until the bit with the backyard philosophy warning.

On the nerd front, I had a bit of a fight with my internet provider as to the stability of my line. They were full of confidence that my line needed to be throttled to 4 Mbit/s to be stable, I had a suspicion that I could get it stable at a significantly higher rate. Guess who’s the daddy? ME OF COURSE. My line is stable at 5.54 Mbit/s (I’m about 3.2 km from the exchange). I now know far more about ADSL line black magic than I ever wanted to.  Bottom line in this case is that, in cases where your line only manages between 4 and 6 Mbit/s, ADSL1 can be much more stable than ADSL2.

Further on the nerd front, my month-long evaluation of Dropbox has culminated in me signing on for a further year with a pro 50G account.  It’s just that good… Even furtherer, lastpass.com has solved my last gripe with Google Chrome, namely that its stored password database is not encrypted. As part of the bargain, my passwords are now synchronized with the Cloud! Coupled with Chrome absolutely crushing most other browsers in speed tests, and the fact that up to now it has proven extremely hard to hack, this has pushed me over the edge, into the Wonderful Land of Full-time Dedicated Google Chrome Addicts.

Warning: Excessive Backyard Philosophy Beyond This Point

On Wednesday I arrived home, all stressed out about work. Yes dear readers, some academics, like me for instance, sometimes stress out about work. Somehow, I was born without the ability to separate work from personal life. Mostly, this is an advantage: I really love my job. It’s become more a way-of-life, a philosophy if you will, than just a job. However, sometimes this means that I have great difficulty switching off my work circuits when this is necessary, for example when work finds itself in High Stress Mode (HSM) and I need to be in Relaxed Family-Person Mode (RFPM). At the end of such a work day, my body is displaced from work to home, but the contents of my head remain exactly the same. In effect, I travel through space, but not time.

Back to Wednesday: Fortunately, fate conspired to arrange for me and my BBQ to meet, under a lovely early Spring sky. My BBQ is more than just a BBQ. I have recently come to the conclusion that it is a powerful magical artifact that is able to bend time (and space). Standing there admiring the glowing coals, everything came to a peaceful stand-still. Probably due to the severe time-warping slowing down the world, I could see the whole multi-dimensional life-landscape surrounding me in its true perspective.

It’s not all that.

Work is really important, and if that’s your passion, it should remain so.  It’s definitely mine. However, there’s more, and much of the more is even more important. I’ve made  this observation often enough before, but it’s one of those things that is apparent to my cognitive self, but somehow doesn’t have a permanent effect on my natural perception. Briefly on Wednesday evening, this realisation was clear. By Thursday the feeling was starting to disappear again.

Epilogue

Today I received the terribly sad news that a colleague, with whom I’d been working closely together for the past months, had unexpectedly passed away on Sunday, leaving behind a young family.

One has to learn how to keep things in perspective. Life is too short not to.

Weekly Head Voices #15: Auto-tune my cloud.

In this 15th edition of my weekly head voices, I move yet more of my life into the cloud, discover (years after everyone else) the delightful auto-tune internet meme and finally go all backyard-psychological whilst staring into the distance, obviously defocused, and waxing on about the purpose of this weblog.

Before continuing, you might like to watch this clip explaining why you shall build a turtle fence (I’ll get back to the clip after my dropbox story):

Last week I completed, you guessed it, 21 GTD tasks spread over 10 projects. Once again, one more task than last week. The question is thus not if, but when I’m going to have to disappoint you. :)  Worth mentioning on the miscellany front is that I’ve started playing around with processing, a fantastic little system for programming visual effects and interaction, in preparation for a new first year course. My goal is to get the students irrevocably addicted to the coolness that is media processing! I’ll keep you up to date…

In a previous post, I was quite enthusiastic about Dropbox and its possibilities for collaboration. As some of you might now, I really like this whole living-in-the-cloud idea: I use GMail, Google Calendar and Google Documents quite extensively and I’m even paying for extra storage with the big G.  So, during the past week, I decided to bite the bullet some more and to move 12G more of my data right into the cloud, courtesy of a 50G Dropbox Pro account.  Up to now, I had a ridiculously complex synchronisation system keeping various subsets of my data up to date between a netbook, a laptop and three different servers. At the core of this system was unison, a brilliant multi-way open source synchronisation tool. In spite of this system mostly working, its complexity and the starkly contrasting It-Just-Works nature of Dropbox convinced me to give the simple solution a shot.

So far I can only report that I remain impressed: At one stage I manually copied a complete dropbox (12Gigs) from one already synced machine to a fresh target machine and started the Dropbox software on the target. It politely asked:

There is already a folder in your home folder called Dropbox. Do you want to merge all the existing files in that folder into your dropbox?

After clicking on the “HELL YEAH!” button (that’s how it felt, ok), the software went on indexing for a minute or two and then correctly claimed that everything was nicely synced up. Very much understated robustness, kudos to the developers. I’m going to test-drive this whole business for one month, and then let you know whether it’s going to be  a permanent fixture in my cloud-home.

Still wondering why you should build a turtle fence? Well, you can blame the Auto-Tune internet meme. Very shortly, auto-tune is an audio effect that corrects one voice to be perfectly in tune with backing music. In other words, a vocalist who can’t sing is in fact no problem at all, computer will fix! Initially it was used quite sparingly and its application was even sometimes kept a secret, until artists such as Cher and especially T-Pain turned it into an art form, in fact exaggerating the effect until it gave a decidedly unsubtle robotic voice effect. The effect has become so famous that it now gets to call itself an internet meme and is often parodied.  The turtle clip above is just one of a whole series (auto-tune the news, see them all!). In the clip below, internet scientists *ahem*, including the well-known Professor Weird Al Yankovic, take an in-depth look at this phenomenon:

Finally, back to the purpose of this weblog… Good blogs all seem to have some central theme, such as photography, environmental issues, science or pokemon. I seem to recall that I’d also seen this in more than one “how to become an A-list blogger” guides.  I don’t find it hard to believe that this is very important.  However, this blog has never had a central theme, it’s always been me blabbing about the various things that I find blab-worthy.  I’ve never been able to come up with something better, and it was definitely not for lack of trying.  The Weekly Head Voices, by focusing my blabbing into slightly more coherent episodes, have finally helped me to come to a conclusion.  Besides acting as a creative outlet, sitting down every week and carefully externalising a specific subset of my experiences with the express purpose of having it read by a small number of people, is an important ritual during which I am forced to distance myself from the events of the week, and to self-reflect.  By formally concluding the previous period in this way, one has the mental room to manoeuvre in preparation for the next. If you by any chance find any of it entertaining, or at least you just can’t look away, it’s a win-win situation!

In other words: Theme-schmeme! The voices in my head will continue to be the many and various topics of this weblog, thank you very much. :)

Have a great week kids!