Sam Smoot

I read this today:

Rails is not bloated it’s modular

It occurred to me that modularity is a crutch. If you really grok loose coupling, you don’t say something is modular.

Modular just means it’s got modules. It’s name-spaced. Bit’s of code with different responsibilities are in different places.

You might gloss over it, like I generally do, and infer that when someone says modular they’re really referring to Separation of Concerns, or Loose Coupling or High Cohesion. You know, real OOAD shit.

Then I realized… I used to use the term modular myself. I didn’t mean any of those things. Even if I had an idea of them, and thought that’s what I was going for, I couldn’t Design my way out of a wet paper bag. What I meant was more that things appeared organized more than anything else; and that must be a sign of good code.

Just some food for thought.

I’m working on Harbor 1.0. Here’s my CHANGELOG so far:

1.0

• MINOR: Request#env[“APP_ENVIRONMENT”] no longer set. Use config.(test|development|production|etc)?
• MINOR: Request#environment no longer exists. Use config.
• MINOR: Harbor no longer serves public files directly. Use Rack::File: eg. Rack::File.new(“public”)
• MAJOR: Harbor no longer serves public files from other applications. Copy those into your app.
• MINOR: Request#application no longer exists
• MAJOR: Events#raise_event2 has been renamed Events#raise_event. The previously deprecated #raise_event has been removed
• MINOR: Harbor::ZippedIO has been removed
• MINOR: Hoptoad Notifier has been removed
• MAJOR: DataMapper Sessions have been removed. Use DataObjects Session instead if you need database sessions
• MAJOR: Harbor::Cascade has been removed. use “run Harbor” against Harbor::Applications instead
• MINOR: Harbor::Mailer has been moved to Harbor::Mail::Mailer
• MINOR: Harbor::MailServers has been moved to Harbor::Mail::Servers
• MINOR: Harbor::MailFilters has been moved to Harbor::Mail::Filters
• MINOR: Removed Harbor::Script

Harbor is a Ruby web-framework we wrote and use at work. It’s about 3.5 years old at this point and it’s been open source for about as long.

Pretty early on it solidified into the framework it is today. Very loosely coupled. Very little in the way of convention. For most of it’s life it didn’t even have a central concept of Configuration or Environment. That was something you added yourself with a case statement over ENV["ENVIRONMENT"] or similar.

In all that time breaking changes have been kept to a minimum. So much so that we recently took a large project that was previously on a version of Harbor over two years old, deployed it to a new box, and since the Gemfile didn’t specify a specific version of Harbor, it was upgraded to the latest, and it caused zero breakages. This project is pushing 100 tables in it’s database schema and round about 500 individual Views, so it’s no little weekend app.

A true test then to the stability of Harbor. Which was the number one reason for writing it in the first place after having the rug pulled out from under us a few too many times before with other frameworks.

That said, I’ll be the first to admit that the lack of a “Golden Path” in Harbor has caused unnecessary pain and confusion. When your options are wide open, there’s a lot of opportunity for doing things wrong. Repeatedly. Until you find what works.

That changes now. Harbor has been “v1” quality in spirit if not in name since forever. Now’s time to make it official.

Before we do though, it’s finally time to break a few eggs, codify the conventions that work, clean up the cruft and occasional deprecation warning, and perhaps most importantly, steal from among the best of what’s out there in Ruby land.

Harbor is among few Ruby frameworks out there I’d say that’s actually an extraction of efforts on numerous projects. Excepting Rails Engines for example, it’s the only Ruby framework I’m aware of that allows you to arbitrarily mount any number of applications together, and overwrite just the views of other applications. So it’s got some things going for it definitely.

Until recently, configuration was not one of those things though. Harbor 1.0’s Configuration object will be pervasively used throughout the framework, giving you sane defaults at every opportunity. Four core environments are baked in:

• Development
• Test
• Stage
• Production

So going forward, Production will cache compiled templates by default for example. Development and Test will filter emails with stubs for testing. Everything but Production will log verbosely. You get the idea.

One of the biggest changes is that up until now, Harbor used PORO (Plain Ol’ Ruby Objects) for Controllers. It’s nice in theory, and it’s good for testing, but we’re about to get a whole lot more Opinionated up in this piece. Don’t Unit-Test Controllers. Just don’t. If your actions are complex enough for it to provide any worth it’s time to refactor. Just Integration Test with Rack::Test or something.

The new Harbor has a long overdue Harbor::Controller class. You might think foisting inheritance on you spoils the party, but the truth is, after writing literally hundreds of PORO Controllers in Harbor, I can’t remember a single one where we used inheritance to factor anything out. So being pragmatic, inheritance it is. Harbor is still using an Inversion of Control Container at it’s chewey center, so you can still do things like get an instance of a mail-server just by slapping an accessor onto your controller. The new Controller base class gives you a few things standard though, like request and response accessors, and a new Router.

The old router looked a lot like a flavor of Sinatra. Except uglier. The convention on instantiation never really made much sense. It worked. We copy/pasted. The end. The new Harbor::Controller class fixes that. It’s all the niceness of Sinatra, without the configuration or single-file-ness.

One thing to note above is that routes can either be relative to their controller, or rooted with a leading /.

I think that’s enough of a sneak peak for now though. We’re getting close. Another couple weeks and this’ll all be buttoned up with an official release. At the end of the day it’s all about providing that Golden Path for development. Sane defaults for everything, but letting you break away when you need to.

If you have any questions or suggestion, feel free to ping me on Twitter! G’night yall.

CARP stands for Common Address Redundancy Protocol. Put another way: An IP that can be shared among multiple machines for redundancy, only one being the Master at any given time.

A CARP interface works by broadcasting an ARP packet that basically tells the whole subnet “I’m here!”.

There are two variables that determine how frequently this happens: advbase and advskew. Meaning Base Advertisement Rate and Advertisement “Skew” respectively.

Advbase is in seconds. So advbase 10 means “broadcast an ARP packet every ten seconds”.

Advskew is a priority to decide who the Master system should be. It can be a number between 0 and 255. This adds a small fraction of seconds of delay to the broadcasting of packets.

Let’s say you’ve got an advbase of 10 and an advskew of 50. Pipe those numbers through this formula:

advbase + (advskew / 255)

You’ll get: 10.196.

So such an interface would send out an ARP packet to say “I’m here!” every 10.196 seconds.

Ok, so that’s your Master system.

Say your Backup system has an advskew of 100 (the advbase for a CARP-group should match, so we’ll keep advbase 10). That machine will send out an ARP packet to say “I’m here!” every 10.392 seconds.

The Master ignores these broadcasts. It’s got real work to do.

Since the Backup system isn’t serving other traffic and doesn’t have anything better to do, it’s constantly listening for the packets from the Master system. Like a stalker.

Say the Master system explodes in a great ball of fire.

The Backup stops seeing the Master say “I’m here!” every 10.196 seconds and panics. This is not a time to panic though. Maybe there’s a bit of packet-loss on your network though. Let’s let cooler heads prevail. You don’t want the Backup to miss seeing just one packet and them jumping up proclaiming there’s a new boss in town and stealing the IP from the Master system even if there’s nothing wrong.

So built into CARP there’s an acknowledgement that it’s not a perfect world, and some wayward packets could possibly lose their way, going off to “find themselves” or some other new-age stuff.

CARP handles this by saying that a Backup starts at denial and has to miss seeing three packets in a row from the Master before it comes full circle to acceptance that it’s buddy has gone to the big server farm in the sky.

Once it misses a third packet it’s game on though. The Backup immediately takes over the IP and decides it’s the new Master. If there’s more than one Backup, it’s a race to see who becomes Master first. Since the one with the lowest advskew will broadcast first, that’s the winner. So if you were to have three systems, you might set the advskews to 50, 100 and 150.

And that’s pretty much it. At the end of the day it’s actually pretty straight-forward. All this just to explain why bringing up a new CARP interface on your Master system with an advbase of 10 and an advskew of 50 will take right about 30 seconds to come out of Backup mode.

This is a follow-up to my Blog post.

I’ve probably put close to 8 hours into my new Blog at this point. Much of that was figuring out pretty forms with Bootstrap, looking up all the various bits and pieces for Sinatra, and deciding how best to store a chronological Set in Redis.

The last bit had me confused. Since the only data-type that can be filtered server-side in Redis is a List, that seems like the only appropriate way to store a series of Blog posts for example. It just felt like I was missing something…

Anyway, I haven’t preserved the old URLs. Or added Categories back yet.

Frankly, I think I’m going to rip out Redis as the main backing store and go back to PostgreSQL. Redis is great, but I’m on the fence about using it this way (as a general-purpose database). It feels a bit clunky and unnatural. For now I think I’ll stick to using it for things like caching, session storage, auth-keys, etc. Which by the way it’s brilliant at.

So we’re not too far off here then and I’ll get back to blogging seriously.

Quick-tip: Including a CNAME file in your Github Pages repository works great even if you’re not using Github Pages for your hosting. Pretty considerate of them. So this blog is now at Linode.

I’ve got some server maintenance at midnight tonight. In the mean-time I’m up here at work twiddling my thumbs with three hours to go. Yeah, there’s work things I could be doing, but I’ve already put in more than a full day and I’ve got a cold. :-p

I’ve looked at HackerNews already.

So I’m going to do something. Write some code.

I like Jekyll. Mostly. It’s just that the whole pull, type-out-an-awkward-filename, write some YAML, write some textile, commit, push dance is enough of a speed bump that I don’t post enough. At least that’s my excuse.

So I’m going to try my hand at whipping up a blog. There’s lots of things I could write it in. Node.js is something I’ve been meaning to give a go. ngx_openresty looks really interesting. But for tonight I’ll see what I can accomplish in three hours and stick with Ruby. I could go with Harbor, but I like Sinatra’s router, and I think this is something I can manage in a single “controller” so I’m going to give that a shot. I could persist to PostgreSQL and Sequel which is always a winning combination. But I think I’ll go with Redis this time around. It’s fast, and ought to suit the blog-post format well. At least initially. Not sure of the best fit, but I’ll try a key for each post, and a List to track/retrieve them.

Follow along here, and with any luck I’ll have something in the next few hours. :-)

I’ve got a disused Linode instance already, so if it works out, I’ll deploy, update DNS and we’ll see how it takes.

‘Tis Bash really. Because it’s the default, or easy to get on FreeBSD, OpenBSD, OSX, Ubuntu and Solaris.

So how many lines is it?

18,865

Yes. I wrote this post just for that. Vanity. Sue me. ;-)

It’s a big collection of scripts to automate server configuration. It spans the gamut from compiling NGinx, to configuring LACP on FreeBSD, to generating HAProxy entries, to cloning git repositories and mounting NFS shares. Pretty much anything you’d need to web-servers, load-balance them, setup replicated database servers, etc is in there.

Why not Chef or Puppet?

Because they’re complex. :-) Force yourself to learn shell-scripting, and soon enough you can fire off a script like the Github Migration in the previous post in a few minutes, saving days of manual labor when you’re migrating over 100 projects. You might be surprised at just how often you use it. I couldn’t imagine doing half of what I used to do manually or with Ruby now.

I mean, Ruby is great in many many ways. But it’s no substitute for being able to fire off a quick for-loop right where your cursor is now.

So if you decide to tackle one new language this year, and you spend a fair about of time on *nix or *bsd like systems, I heartily recommend Bash.

By the way, I named that system’s administration project after this beer. Gubna. ;-)