Build Your Interfaces

layers of tech: hardware to ui

We are in the connected age. People halfway across the planet are only a few clicks away. Pluto, one of the furthest bodies in our solar system, is just as many clicks away.

It’s not enough that we are connected though. Every one of our devices must be too. Your phone, wristwatch and even your lightbulb want to connect with everything around.

A long time ago, you would have needed a deep understanding of every layer of technology and how they inter-operate to tinker, let alone build anything meaningful. These days, the hard parts have been taken care of and are presented as simplified APIs.

You don’t need to understand the intricacies of communication with Bluetooth. Your device probably provides access to the Bluetooth API, so you can harness it without needing to understand the lower levels of it.

With our devices containing more technology and the barriers to harnessing them getting lowered everyday, we have the unique privilege of being able to build our own interfaces for our devices.

I track my clothes to make it easier to simplify my wardrobe, and to guide my purchases.

I started by writing down dates and clothing combinations on an index card in the morning. Since my index card was unlined and my brain is pretty fuzzy early in the day, I occasionally had problems with incorrect dates or items not matching up. Eventually, I built a small Rails application (Quantified Awesome) to keep track of the clothes for me. Adding pictures made it easier to select the right item. Over time, I added little conveniences like the ability to display or sort by the last time I wore something.

Sacha Chua

With a little bit of training, we can harness the awesome power of the devices in our pockets and on our wrists. No longer constrained to interfaces the original programmers and engineers created, we can design how we want to interact with technology. With a little bit more effort, we can weave separate technology together and control them with a single interface.

A friend of mine needed to watermark images for her blog, so I built this interface for her.

I know little to nothing about how image manipulation works on a low-level, but it was trivial to stitch a provided API together with web technology to create an interface she could access from her phone or laptop. Select your pictures and they get automatically watermarked.

Don’t be limited by how technology is presented to you. Create your own interfaces. Shape your world.

 

Why Fashola’s Website Costs 78 Million Naira

Many Nigerians are upset at the news that Lagos State spent N78 Million to maintain a website for their governor.

I can see why many are confused and outraged, considering that blogs are hosted for free on WordPress.com, great dedicated servers cost roughly N2000 a month and a fraction of that budget would retain talented Nigerian staff members to keep the site up to date for his tenure as governor.

As a professional web developer and a Lagosian, it is my duty to help people understand why N78 million is not just a reasonable cost, but actually a bargain. At the end of this, I hope I’d have convinced you, not confused you, that Lagos state was financially prudent in this project.

Continue reading Why Fashola’s Website Costs 78 Million Naira

My First Accepted Pull Request

In the process of creating my first microservice, I used RabbitMQ to create a reliable job queue. I’d never used it before and it’s pretty complicated so I needed to find a library with a simple api for getting started with it.

The library I settled on is jackrabbit.

It has drop-dead simple api which hides the complexity of working with RabbitMQ. Perfect for the first-timer who doesn’t know the nuances of queuing strategies.

While I was monkeying around with it, I noticed my queue wasn’t actually queuing. Messages that don’t get immediately processed disappear without a trace. That sounds like the very opposite effect of what I decided to use RabbitMQ for in the first place!

After some sleuthing, it turned out jackrabbit sets messages to expire after a second, and there was no api access to configure it. You can’t increase the duration or remove it. One second messages and that’s that.

Coincidentally enough, jackrabbit’s maintainer exposed the api a few days later, so I didn’t need to switch to a more complex library.

Since the default was still one second and it seemed like quite a few people wanted their messages to hang around in their queues, I decided to update jackrabbit docs to show how it’s done.

And he accepted my updates!

https://github.com/hunterloftis/jackrabbit/pull/16

Officially my first contribution to someone else’s project. Sure it’s not functionality or a fix, but it’s better than nothing.

Can’t wait to make bigger and better contributions 🙂

My First Microservice

Last year I built a small app for a blogger friend of mine. It saved him the trouble of downloading a file just to rename it before uploading to the blog. He just supplied a new name, a link to the file, pushed a button and it was automagically done.

It saved him money because he didn’t to consume his data plan downloading the media files he needed to host for the blog.

It was also much faster than doing it manually. I tested it with a 500MB file and it was transferred in 19 seconds. No way he could have downloaded, renamed and uploaded that fast.

(There’s a video of it in action in this old post)

Not long after it was built, I started toying with the idea of setting it up as an app others can use. It took a few months to settle on core features and the architecture for public user but I’m finally happy with the details.

The original was architecturally a monolith. The http server, ftp client and socket.io server were all in the the same file. While it was fine for a single user or a team, it would have started showing signs of stress as soon as it was opened to the public. It needed to be decoupled.

In my defense they weren’t that tightly coupled together. I used an event system to keep each part independent while in the same file, so it was relatively easy to see a path from my monolith to a microservice.

I started by pulling the ftp server component out. It was an obvious first choice because that would be the most process-intensive component. Pulling it out would give me the most breathing room architecturally. I can pull the rest apart as they outgrow the monolith.

The first challenge was keeping all the separate components in sync.

As a monolith, the user’s request to push initiated a socket.io connection which communicated the progress of the push job.

Now that the ftp push is on its own server, it needs a way to communicate with all the components reliably. It needs to collect jobs from the http server and send progress updates to the socket.io server.

I moved the communication between components from an event system over to RabbitMQ and Redis.

FTP push requests are put on a RabbitMQ queue for the ftp server to pick up from. It provides a lot of assurances, like a request won’t be lost no matter what. If the server fails to push it, the request is put back on the queue. The queue is durable so it will persist even if RabbitMQ crashes.

It would be overkill to send progress updates over RabbitMQ, so I use Redis pub/sub to communicate updates to the socket.io server, which sends them to the client.

Of course the monolith needed to connect to Redis and RabbitMQ. That was trivial though, since using events isn’t dramatically different.

I’d learned my lesson about testing, so I needed to modularize the ftp server for testing.

I created an api that abstracted the ftp server’s abilities to receive jobs and send messages and exposed them to the test runner.

At this point, everything is feature complete in comparison to the original monolith, with the added benefit of the most intensive process isolated, independently scalable and tested!

I’m still a little ways away from releasing the app, but I’m really loving this microservice architecture so I’ll continue building new components in this pattern.