Basket: Turning machines into text for fun and profit

Lately I've been rebuilding a lot of servers and workstations, and so one of the many things I've been trying to find the time to work on is basket.

Originally started as one of those random in-office projects, my coworkers and I have taken it upon ourselves to extend that flaky set of bash methods and maybe turn it into a useful set of low-level server administration utilities. It's difficult to say yet how much functionality we'll provide - the project is very much in its infancy - but it's starting to look like a sensible and robust enough idea that I'm posting about it here. From the readme:

It's a bash thoole ket, or bash toolkit if you're sober. Yeah, we tried. It's also a basket of bash functions so that kinda works.

Basket is like a basket because you can dump little scripts into it with no overhead other than storage. When sourced, only the main (small) basket of bash helper methods is loaded - special-purpose modules are brought in on demand and have a simple dependency management system. (So if basket ever gets too big, you can just blindly delete stuff you don't need out of the lib folder). (more)

Injustice, Ethereum and the information renaissance

Do you ever stop to think about the ways we communicate?

When you talk over the phone your call can easily be monitored or recorded. Your private life is potentially exposed, but just for a brief moment. No big deal, right? Nobody is going to be listening to you unless you're doing something wrong. But the call itself and the fact that it happened is retained for a few years by your phone company. Thus that window into your life is widened, just a little bit.

When you send someone a text message the person you're texting, your location and the message itself are also retained for a few years by your phone company. When you visit any website or use any app which requires an internet connection the same thing happens and could soon be mandatory. The window widens again.

When you chat on Facebook literally everything you're saying and in some cases what is going on in the room around you is recorded. Forever. If not by Facebook then by the governments tapped into the internet backbone. When you use Google Hangouts you record your face, what you're doing and the environment around you and hand that over forever too. It happened with Yahoo Chat, and it will happen again. When you sync your files to Drive or iCloud or OneDrive or Dropbox, you give away the very real and substantial value of the contents of your hard drive. When you sit in front of a Kinect, you invite Microsoft and probably the US government into your home to study you in intricate high-def three-dimensional detail. When you post and tag those photos of your night out on Facebook or Instagram you provide structured information to assist a powerful AI in enhancing its detailed 3D maps of you and your friend's faces and predictive models of your behaviour. And when we start wearing Google Glass around and buying devices like the Amazon Fire, we will be handing over the world around us in real-time.

Truly. Legitimately. Non-crazy like. As described in the NSA's own documents as “Collection directly from the servers of these U.S. Service Providers: Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple.” Petabytes of it every day. Handing over our data until, as Google's CEO said, “We don't need you to type at all. We know where you are. We know where you've been. We can more or less know what you're thinking about.”

It's as if we want to do away with the window completely. (more)

This is your government on metadata (with pictures!)

Data retention and warrantless government spying are starting to come out as two of the biggest issues of the century. And about time too - this is a war that has been going on for decades, hidden behind the screens of software developers, tech geeks and hacker kids where everything looks technical and boring to the outside observer and prevents anyone from really paying much attention to what's going on.

There is a lot being said about what our representatives are trying to legalise here, but what is already legal? What is already going on behind closed doors?

The issue has been raised in Europe - and rejected as a violation of basic human rights. It has been raised in England - and allowed to happen against the wishes of the people. It has been raised in America - brought to light after it surfaced that the American government has been secretly storing communications from the entire internet despite this obvious conflict with their constitution. Now the Australian Government is having its turn at the grab for more power. (more)

Terms & conditons, tracking and choices

It alarms me setting up a new phone.

Once you have linked your online accounts, whether they be or or, you get to the annoying setup wizard that we all loathe and rush through. The screens wherein we mash 'next' impatiently so we can get to the end and start using it.

Nobody pays any attention to these things, everybody wants the convenience of knowing their online stuff 'just works' and that they won't lose their emails. But what is the real cost here? (more)

Git deployment script

Deploying your projects via git (or any kind of version control, for that matter) is a no-brainer. It's been discussed many times (this is probably my favourite) and as we all know, it takes all the manual labour, time and human error out of the process.

You don't often find deployment scripts though. Perhaps it's just me failing at Google, I don't know; but perhaps people just don't tend to write them generic enough. Or perhaps they are too simple to be bothered sharing them. In any case, I didn't like the idea of writing single-use shell scripts and so I whipped up something that does the job rather well, and in a generic and reusable way. (more)

Responsive web design and layouts

Recently I've been doing a lot of responsive sites for clients. They're fun and often challenging things to work with; opinions and solutions from various developers and designers scattered liberally throughout the internet. So I suppose it's about my turn to chime in on it all. (more)

Percona Cluster on Ubuntu 12.10 (Quantal Quetzal)

Recently it's been a task of mine to setup a new SQL database cluster. My favourite database engine of late has been Percona's XtraDB, and so naturally I was quick to choose Percona Cluster as a solution.

This installation was a Cluster 5.5.28 install performed on a RackSpace public cloud server using a clean Ubuntu 12.10 image- your mileage with other platforms, OSes and Percona Server versions may vary. (more)

A good Webalizer user agent grouping

Recently I had to go about setting up my own configurations for Webalizer. Grouping user-agents into meaningful sections seemed a worthwhile cause, so I ran through the mobile user-agent string list and setup matches for mobile devices on top of some of the more straightforward ones for desktop. The result is a pretty clean config that handles (in my experience) about 90% of my incoming traffic. I've also tried to provide a little bit of forward-planning for version numbers on some of the browsers which now update their major version almost daily (curse you, chrome), but you should plan to keep that aspect up to date yourself. (more)

Google Apps email administration guide

Whilst they offer some great free services (edit: Google Apps is now only offered on paid subscriptions, even under the previous 5 user cap. But if you got in beforehand, you do get to keep it for free...), Google's products with their user-friendly interfaces can occaisonally be confusing for those of us who know what we're doing - a whole new set of terminology to learn just to be able to do what you had previously learned in the 'normal' / 'homebrew' / whatever way.

This is a quick cheatsheet on how to achieve some of the fundamental and more advanced mail behaviours and operations that you'd expect from any mailserver via your Google Apps admin console. (more)