Configuration management contains information that shouldn’t be floating all over the place in plain text (like your root db credentials).
Within Chef, metadata can be stored in a role, cookbook, attribute or data bag. Although, only a data bag can be encrypted, making them perfect for storing sensitive information.
If you are using knife solo you will need the following plugin installed to get started:
gem install knife-solo_data_bag
Next, set an editor environment variable. So knife can spin up your text editor of choice when editing encrypted data bags:
A key (password) locks and unlocks your encrypted data bag. I like to defined this in a file named ‘data_bag_key’:
echo “super secret password” > data_bag_key
Make sure you include your data_bag_key file as part of your .gitignore file. No point in locking everything up, if you end up taping the key to the front of the lock.
To create an encrypted data bag with the knife command:
knife solo data bag create credentials production --secret-file 'data_bag_key'
Editing an encrypted data bag is also done with the knife command:
knife solo data bag edit credentials production --secret-file 'data_bag_key'
Viewing encrypted information in done with:
knife solo data bag show credentials production --secret-file 'data_bag_key'
Using encrypted data bag information within your cookbooks is a little more involved. So I created a little utility credentials cookbook to make things easier. Place ‘recipe[credentials]’ at the start of your run_list to decrypt the credentials data bag. The resulting metadata is automatically added to the node attributes.
With some sensible data bag naming, cookbook default attributes (like passwords) can be overridden.
For a while now I have been trying to learn a little about functional programming. I was lucky. I went to some free courses run by NICTA and Tony Morris. I was also able to hassle Tony about concepts I didn’t understand. It was one of the most confronting experiences of my professional life. The subject material wasn’t easy.
Then last year, a couple of former colleagues launched a ‘MOOC’, The Science of Everyday Thinking.
It was an excellent course that helped me debug the way I thought about the world. Episode five, ‘Learning to Learn’ stood out the most. Actually it made me stop.
Shit. I am an expert beginner.
During my time at university I managed a bit of a fluke. By accident, I developed effective learning techniques (something not everyone does). I didn’t know it at the time, and I certainly didn’t appreciate it.
I found my internal monologue deriding people ‘That ain’t gonna help you.’ whenever I saw people furiously highlighting textbooks, and transcribing lectures. And, by the end of my university degree I had become so cocksure of myself. I thought I could learn anything. Give me three weeks, the course material, past exams and I could better than pass anything you threw at me.
By the end of my degree, I refined The Study Ritual. Two or three weeks out from the exam period I would withdraw from the world and begin.
It started with a timetable. Two or three practice exams for each of my courses, interleaved over those weeks. Authentic exam conditions. Usually the kitchen table, timed, with only what I could take into the real exam. I can assure you, I ‘failed’ every first practice exam I did during The Study Ritual.
Each day, I had forty minutes of revision allocated to each subject, with a twenty minute break in-between. At the start of The Study Ritual, I revised by creating course summaries. Towards the end, I did basic retrieval practice. This practice was either working through example problems (like the ones I flunked in practice exams) or that classic ‘cover, write and check’ technique for remembering stuff.
It took a fair amount of discipline and work. But by the time an exam rolled round, I was confident. A few butterflies in the belly, but I didn’t seem stressed compared to some of my peers. It also worked well. My performance at university steadily improved as I refined The Study Ritual.
So what happened? Why was I finding this functional programming stuff so damn hard?
During a professional career, you often optimise on performance, not learning. As a software engineer I am evaluated on how quickly I can build quality systems. Not how well I understand the material I use to build these systems directly.
The problem? I was able to build systems well enough to get a few promotions. But, I wasn’t building underlying knowledge fast enough. How I build knowledge in my professional career is not as effective as The Study Ritual. Functional programming was the first desirable difficulty (the learning equivalent of no pain, no gain) that I had faced in five years.
In many ways, I hope my monastic engineering experience will address this problem. Will I be able to develop better learning techniques that are more compatible with the time pressures of professional work? Will the one month projects I build at the end of my engineering experience be better than those at the start?
Today I ‘finished’ up at my day job. Why the quotation marks? We’ll get to that. But first, a little context, as this is a day I have been working toward for five years.
I have always been a restless employee. Despite working in great environments and on cool projects, I was constantly chasing the technology dragon. I always had a side-project or two on the go outside of work.
Often I had lofty ambitions for these side-projects; I wanted to create a business. My business. Something I could control and direct, something that would allow me to find a greater purpose.
When pursuing a startup or business, thoughts of fortune, fame or freedom are often lurking in the back of one’s head. During each side-project I would think, ‘This is the one, this is how I will catch that elusive break’. But it never happened - all I appeared to do was notch up an impressive collection of failures and setbacks.
It wasn’t till a couple of years ago that I realised that each new opportunity I picked up was a direct result of a quirky side-project. Irrespective of how I felt I had failed, I would land a new gig and get to work with new technologies alongside even more skilled people. But I was still chasing the technology dragon. It wasn’t enough. There was always ‘the next tech’ to use and another bunch of talented people I could be learning from.
I wasn’t notching up failures. I was slowly ‘grinding’ and levelling up on a strange massively multiplayer online game called professional software development.
With this epiphany, I decided to dig deeper into my own startup dreams. Was having my own business what I wanted? As it turns out, no. I was after the freedom to create.
I hinted a year ago, I have an interest in exploring monastic engineering experiences. I want to bang away on my keyboard and create, just for fun, without any distractions. Not to create a business, not to make products that people want, not to discover something new, not to publish a paper. But to indulgently explore oddities that perhaps only I find interesting.
So I have set myself a goal to build one project a month for twelve months. The only rule? I must be able to build it in a month. If I manage to build twelve projects, only to have someone ask, ‘Clinton why on earth did you … ?’ and my only answer is, ‘Because I could’, I will have achieved what I set out to do.
Armed with my insight and plan, I went to my boss to try and resign. When he heard what I was doing, he looked me in the eye and offered me a job. “Will you stay on and work casual for me?” My boss made it sound like I was doing him a favour, but it was the other way round. He was offering a safety net to work as much or as little as I needed - why? Because he is a bloody good boss, and was starting me on my journey. He was removing a distraction, the fear of losing a somewhat valuable career.
Phew. It has been a ton of work to get here. To get to the beginning; a long summer break before starting my project-a-month pursuit of programming nirvana in the new year. I have no idea what will unfold, but it is going to be fun finding out.
To everyone who helped get me here. Thank you.
So for a long while, powering prototypes has been a bit of a chore for me, I was constantly wrestling a desktop Kraken of cables. A giant mess crawling out of the umbilical of an old ATX computer power supply, and a couple of wall wart style USB chargers (for powering Arduinos, Phones or Raspberry Pis).
The BK Precision 1550 is a low-cost benchtop power supply that is capable of generating 1 to 36 volts, with an output maximum of 3 amps. It retails for about $135 on amazon. However people outside the U.S. need to be a little careful, the standard model only accepts 100-120V. There is a 220V model (compatible with the mains voltage here in Australia), you just need to hunt around for it, I picked mine up from Mouser Electronics.
I was pleasantly surprised when I unpacked the 1550, it has a really small footprint and is incredibly light. The industrial design of the faceplate is a direct riff on a black iPod Classic, featuring click wheel inspired control buttons and a large back lit LCD screen.
However, just about everywhere else the BK Precision 1550 was good enough, but seemed to agonisingly fall shy of being an amazing power supply. The team at BK had some great ideas in this one, targeting it at the maker/hobbyist/artist/hacker market, but was ultimately held back by small little details/niggles.
For example, the USB charging port drew my attention first. Hopefully I would be able to ditch a few of those wall warts. Unfortunately it has a max output of 0.5 amps. Enough to charge a phone, and run an Arduino. But not enough for the Raspberry Pi/Beaglebone Black. I would be doing my best Jim Cramer buy impersonation right now if BK had sprung for that extra amp and brought the output of the USB charger to 1.5 amps. Running a Pi off the charging port, and simultaneously being able to power a 12 volt circuit the Pi was controlling? Yes Please.
BK also didn’t include any display of the current going out on the charger port, no big deal really and it certainly won’t stop me from using the USB charging port. But getting a breakdown, and able to see total power usage across everything? Hells yes.
For a cheap power supply, it is also accurate enough at +/- 50mV. However, the controls only allow 100mV increments, and the LCD display also rounded the voltage output going out on the banana plugs. This effectively made it impossible for me to configure the the 1550 to read 12.0 volts. Set it to 11.9, and the old fluke gave a read out of 11.94V. Cool. Press the + button once to add (100mV) and the unit read 12.1, and the fluke 12.05 volts. No matter what I did, I couldn’t get the 1550 to land on 12.0v. The output of 11.9 and the 12.1 settings were close enough to 12 volts for my purposes, but it did trip my OCD out a little not being able to get the display to read what I wanted.
Instead of rounding the output voltage, BK could have just used the ceiling or floor values instead. This would create a greater feeling of control over the output, by being able to hit every 100mV increment between 1 and 36V with no appreciable loss in accuracy. Still 12.05 volts coming out, totally close enough to the 12.0 I am trying to get out of the unit. Why not let the LCD display say - hey this is 12.0 volts… Ish?
These are just tiny tiny little details that don’t make this a bad power supply at all. It is a solid unit that is vastly better than the kraken of crap that was sitting on my bench. Overall? 3.5/5.
Hardware for my next project has started to arrive, and last week a preview kit of Estimote iBeacons landed on my doorstep. Packed like a fine box of Belgium chocolates, the Polish made iBeacons looked delicious.
In site-specific theatre performances, I often use virtual beacons by nominating the coordinates of a target location in software (often to trigger audio or video) and then using GPS to work out the proximity of the participant to the virtual beacons. Naturally, using GPS in the measurements mean it is not very accurate and the proximity calculations could be out a far as 15 metres.
To test how the Estimotes stacked up, I wrote a little ranging test app with their SDK so I could measure the distance between a phone and one of their iBeacons.
I laid out a tape measure, put my phone (A Sony Xperia M2) at one end, and the beacon at the 1 metre mark. Took 30 distance readings from the test app, and repeated at 1 metre increments till running out of tape measure. *
I then used a common GPS accuracy measure, the Distance Root Mean Squared (DRMS) to work out the ranging accuracy. If you are after the raw measurements and stats, you can check out the full dataset and calculations here.
As hinted by the Estimote documentation, I was expecting precision to drop as distance between the phone and the beacon increased.
Which is exactly what I got, the further away (and the weaker the bluetooth signal), the less accurate the distance measurement became. The above results are probably a ‘best case’ scenario as well, I imagine obstacles and anything that would interfere with the bluetooth signal would lead to an even more radical drop-off in accuracy. You can see at 8 metres, the estimote gave a mean reading of about 7 metres, but could be off by at most 2 metres, with the calculated distance varing from 9 to 5 metres.
Estimote DRMS Results
||Mean Measured Distance
What was startling is how accurate the Estimote is at very close ranges, around 10cm when a metre from the iBeacon. But even to be accurate within 2 metres at a decent distance from the iBeacon was vastly better than what I was getting with GPS and virtual beacons. So, I’m definitely going to continue pushing ahead with the Estimote on the next project.
* The broadcast power for the Estimote iBeacon was left at the factory default, (Weak, -12 dBm) which is good to about 15m. I only had 8m of tape measure, so it seemed like a pretty good place to start.
SD cards have a limited life, and the more you read and write to them, the shorter their lifespan. In a Raspberry Pi this makes things a little tricky, the SD card gets a much tougher workout than it normally would in something like a digital camera.
Up until now, I have always just picked up a SD card that was sold bundled with a Raspberry Pi. Unfortunately these SD cards are often low quality and didn’t last very long, with some failing in as little as a month.
So now I get SD cards separately, and only ones that feature wear levelling. The cheap SD cards don’t have any wear levelling, and the Raspberry Pi gets into situations where certain areas on the SD card gets written to over and over again until it wears out and fails. The Pi then comes along and again tries to the use the same worn-out area, and promptly chokes. All despite other areas of the SD card being hardly (or not) used at all!
The more expensive cards with wear levelling won’t just keep pummelling the same spot on the disk over and over again. Instead, it will try and spread wear out over the whole disk. A little like rotating the tyres on a car, wear levelling ensures that each part of the disk decays at about the same rate.
I also get SD cards that have way more space than I need, at least 8GB. This further increases the longevity of the card by increasing the total ‘surface area’ that will eventually wear away. With wear levelling, more free space means a longer lasting SD card.
The following SD-Cards all feature some form of wear levelling and won’t break the piggy bank:
The newer Raspberry Pi B+ takes MicroSD cards, the following all feature some form of wear levelling and won’t break the piggy bank:
However, if you have more than $150 to burn, you can reach to the very top shelf and have a browse around Panasonic’s industrial grade SD cards (some feature RAID for even greater data protection).
The impressive range and features of the Panasonic industrial range has me a bit smitten for the Panasonic gold series. Some of that industrial goodness has to be rubbing off on the top-end of their consumer line, right?
For a host of other tips and tricks on how to extend the life of a SD card inside a Raspberry Pi, see this excellent thread on StackExchange.
2014/10/08 - EDIT: Added MicroSD card list for the Raspberry Pi B+.
Raspberry Pi’s are cheap, and I really like the Raspbian operating system, I can work in just about any programming language I want, and I can configure it from my laptop with standard networking gear. No extra keyboards, monitors or serial adapters needed.
Jack a cable from the Pi’s ethernet port straight into your network router, insert a SD card prepared with Raspbian and power up the Raspberry Pi. 98% of the time, your network will automatically assign the Pi with an ip address (the other 2% being cases where you have explicitly configured your network otherwise).
Next, fire up a network analysis tool and find the address that was automatically assigned to the Pi. For me, I use the excellent Angry IP scanner (open source) to scan my network and get an address list of all the devices that are currently connected.
Now that you know the address of your Pi, you can crank open a terminal and SSH into it with the default credentials that comes with Raspbian:
laptop$ ssh firstname.lastname@example.org
(The default password is raspberry)
Now that we are in, the Raspberry Pi can be customised for your project. I upload and run a little script that installs the tools I use (OpenCV, vim and Go), it also ensures that the operating system is set to handle regular USB webcams and finally configures the timezone.
# Update aptitude
sudo apt-get update
sudo apt-get upgrade -y
# Install opencv & vim.
sudo apt-get install -y libopencv-dev
sudo apt-get install -y vim
# Ensure webcam module is loaded with necessary settings.
sudo rmmod uvcvideo
sudo modprobe uvcvideo nodrop=1 timeout=5000 quirks=0x80
# Download golang
if [ ! -f go1.2.2.linux-arm~multiarch-armv6-1.tar.gz ]; then
# Install golang
sudo tar -C /usr/local -xzf go1.2.2.linux-arm~multiarch-armv6-1.tar.gz
echo "export PATH=$PATH:/usr/local/go/bin" >> /home/pi/.bashrc
# Configure the local timezone.
sudo bash -c 'echo "Australia/Brisbane" > /etc/timezone'
sudo dpkg-reconfigure -f noninteractive tzdata
Often I will tweak this base configuration with some project specific changes, which will customise the networking setup and tweak /etc/rc.local to run a custom application when the Raspberry Pi is booted.
Uploading the bootstrapping script from your laptop is pretty straight forward with scp:
laptop$ scp bootstrap.sh email@example.com:.
Running the script on the Pi is simply a case of:
You can pick up a Raspberry Pi from Amazon and when coupled with an Arduino it is a very flexible platform to power all those creative computing projects you are cooking up.