15 October 2014

The last TrueCrypt 7.1a: cross checking hashes

TrueCrypt has had a good run, and the latest version was intentionally crippled so users can only read from but not otherwise use TrueCrypt volumes anymore.  Internet drama aside, some people have hosted the last full-featured TrueCrypt version 7.1a.

When downloading such security software, you should always check the source of what you download, and this is especially the case with the aftermath of this particular incident.  So I list below some sources to cross check the last version of TrueCrypt.

Open Crypto Audit Project started as a crowd funded way to get a full audit done on TrueCrypt.  They have posted a link to a verified source and binary repository on GitHub under the AuditProject account.  OCAP is explained a bit in this ArsTechnica article, noting that Thomas Ptacek is running Phase II, and Ptacek describes a bit more on this HN thread.

Of course, if you check the files from that repository against the hashes hosted on that repository, you'd expect them to match, even if it was maliciously set up.  So let's cross check with other, hopefully independent and trusted, sources.

Gibson Research Corp. has hosted a TrueCrypt Final Release Repository as well.  Gibson notes the same issue noted above, that you cannot check files against hashes hosted from the same location.  He references a PGP signed file of hashes hosted at Defuse Security.

As discussed on another HN thread, TCnext is hosting another TrueCrypt repository. TCnext refers to a set of "Independent" hashes hosted by German IT-News Golem.de.

At the time of writing, in cross-referencing all hashes mentioned above, the SHA256 hashes were all identical.  Further, the source and binaries hosted by AuditProject on GitHub matches against those SHA256 hashes.  You should check for yourself when you download them, of course.

11 September 2014

Installing Lubuntu 14.04 LTS with Full Disk Encryption

I'm going to walk through, complete with screenshots, my installing Lubuntu 14.04 LTS, a modified distribution of Ubuntu Linux that uses the lightweight LXDE desktop environment and OpenBox window manager.

The last time I installed a fresh copy of Ubuntu was probably when I wrote up Installing Windows 7 & Ubuntu UNR side-by-side on Dell Mini more than four years ago.  Before that, I installed Ubuntu on a desktop, which I upgraded to Lubuntu by installing the required packages but without uninstalling any of the Unity shell items from Ubuntu.

With the new LTS release of Lubuntu, I felt it's ready for conservative users like myself to install.  LTS means it has three years of long term support, which means I don't have to do any major upgrades for at least that long (of course, normal minor upgrades from week to week is still necessary).

A fresh install gives us a chance to clear out the cobwebs, idle packages that were installed but is no longer needed by us, etc.  It also gives us a chance to install it with full disk encryption (FDE), which was available before, but didn't seem quite ready for prime time for conservative users.

Let's begin!

31 August 2014

Haskell Data Analysis Cookbook - a Book Review

As with my previous post, Clojure Data Analysis Cookbook - a Book Review, I was this time offered to review Haskell Data Analysis Cookbook by Nishant Shukla.  First impressions: those are two very similar and related books that have some overlapping ideas, but not only are the programming languages used totally different in "genre", the content itself also cover some different data analysis grounds and could be treated as complementary books in that way.


The book itself is very example oriented (much like the Clojure Data Analysis Cookbook), basically being a collection of code recipes for accomplishing various common tasks for data analysis.  It does give you some quick explanations of why and what else to "see also".

It gives you recipes to take in raw data in the form of CSV, JSON, XML, or whatever, including data that lives on web servers (via HTTP GET or POST requests).  Then there are recipes to build up datasets in MongoDB, or SQLite databases.  To recipes to clean up that data, do analysis (e.g. clustering with k-means), to visualizing, presenting, and exporting that analysis.

Each recipe is more or less self-contained, without much in building on top of previous recipes.  It makes the book more "random access".  It's less a book to read through cover to cover, and more of a handy reference to use by full-text searching for key terms, clicking on the relevant topic in the table of contents, or by looking up terms in the index.  It's definitely a book I'd rather have as a PDF ebook so that I can access it anywhere in the world, and so I can do full-text search in.  It does come in Mobi as well as ePub formats, and code samples are provided in a separate zipped download as well.

Having said that, you can tell whether a book was made to be seriously used as a reference or not by looking at its index.  There are 9 pages of indices, equivalent to about 2.9% of the number of pages previous to the index.  This book can certainly be used as a reference.

As a reference book, it's great for people who have already a familiarity with Haskell in general.  If you don't know Haskell, this book won't teach it to you.  That is, unfortunately, possibly a missed marketing opportunity, as those who don't know Haskell (but have knowledge of another programming language) really only needs a small bit to understand enough of how functions are written in Haskell to pick up what's going on in the book.  This means if you know another programming language, know a bit about data analysis, you could use this book to learn some Haskell so long as you pick up the basic syntax with another tutorial in hand (so it's really not a show stopper to using this book).

Similarly, I'd say you had best be familiar with how to do data analysis as a discipline in itself.  If you don't know whether to do clustering or regression, or whether to use a K-NN or K-means, this book won't teach it to you.

Much of that is, of course, echoing the Clojure Data Analysis Cookbook.  Where the Haskell Data Analysis Cookbook differs, makes the two books have a set of complementary ideas.  Whereas both books talk about concurrency and parallelism, the Clojure DAC goes into those topics (including distributed computing) in much more detail.

On the other hand, whereas both books talk about preparing and processing data (prior to performing statistics or machine learning on it), the Haskell DAC goes into much more detail on topics like processing strings with more advanced algorithms (as in computing the Jaro-Winkler distance between strings, not like doing substring/concat operations), computing hashes and using bloom filters, and working with trees and graphs (as in node-and-link graph theory graphs, not grade-school bar graphs).

So in some sense, the Haskell Data Analysis Cookbook has more theory heavy topics (graphs and trees!), whilst the Clojure Data Analysis Cookbook has more "engineering" topics (concurrency, parallelism, and distributed computing).

Neither books are comprehensive treatise on the topic, but someone who needs a practical refresher on working with graphs and trees may find Haskell Data Analysis Cookbook to be quite useful.

All in all, I'd say this is a decent book, because if you have some familiarity of Haskell, have some familiarity with some of the basic technologies like JSON, MongoDB, or SQLite, have taken a class or two of data analysis or machine learning in university (or a MOOC?), and aren't expecting a lot of hand holding from the book, then this book is a great guide to start you off to doing some data analysis with Haskell.

15 August 2014

Java has deep expression problem for beginning students

There are many problems with Java as the first programming language to teach students if we wish to provide the most effective learning experience.  I've written on this in Learn Python instead of Java as your first language in the past even.  So what now?

Newbie, meet the Expression Problem

Stuart Sierra provides a very lucid explanation of the Expression Problem, a classic problem in software programming, in Solving the Expression Problem with Clojure 1.2.  Needless to say, Clojure provides a very clean solution.

Java, however, is a quagmire and requires some heavy OOP software engineering concepts to solve the Expression Problem.  One wouldn't ordinarily think this has anything to do with beginning students just learning to program though, but it does, and here's how.

Imagine our beginning student, "Sam", starts to learn Java and eventually starts to write a classic game of asteroids.  Sam plugs away and gets a decent game of a single player ship shooting lasers at one kind of asteroids to begin working.  Not bad!  But Sam wants to do more.  Sam wants to not just have one kind of (big) asteroids, he also wants to have small asteroids to shoot at.

Alright, so Sam begins to modify the BigAsteroids class to also be able to represent a smaller sized kind of asteroids.  The teacher catches wind of this and tells Sam, "no, that's not good", and that Sam needs to use OOP principles to write a different class for SmallAsteroids.

Now most students would say "why, Mr. Teach", my way works.  But Sam is a good student and does as he's told.

So Sam goes and creates a second class for SmallAsteroids.  Except his program was built presuming that the only things to draw, to shoot lasers at, and to move around, were BigAsteroids.  None of those methods he wrote to draw, to shoot lasers at, and to move around BigAsteroids work for SmallAsteroids.  hmm...  Welcome to the Expression Problem, Sam.

22 August 2013

Photo album sync fail on iPad with Mac

I'm not usually one to blog complaints about products, but this seems outrageous.  Using the built-in Photo app on an iPad, I've got a bunch of photos organized into albums.  You can download the photos to a Mac in a mass download using the Mac's Image Capture program, or using Mac's iPhoto.  You can backup the iPad's Photo and albums using iTunes, but the backups are inaccessible on the Mac as albums, photos, or files, just pure backup to be reloaded to an iPad in case of "emergency" I guess.  But you cannot download the photo albums to the Mac from the iPad for further use or organization.

Apparently, you cannot download photo albums created on any iOS devices to a Mac for further use or organization at all, and it's been this way since forever.  There is a third-party app called "Phone View" that reportedly can let you do this.

But really?  A simple feature like syncing albums in an Apple built-in app requires a third-party solution?  And it's not obvious when using the beautifully created Apple Photo app on the iPad that album sync was not possible, luring unsuspecting users into creating albums that could not be synced.

This is an especially sad situation for users on iPads or other iOS devices on which they don't have access to install apps.  Where does that ever happen?  Well, corporate and school based usage comes to mind.