featureenvy

The blog of a thinkerer.
By @featureenvy

All rights reserved.

rspec-given Shows How RSpec Can Be Extended

Just recently Jim Weirich released rspec-given 2.0, an extension to the RSpec DSL. It tries to add given, when, then semantics to RSpec. Why? Because given, when, then state intent. So everything in a given will be something that you expect to be present. when is the part that should actually be tested and then states what should happen. This makes it easier to see what is just a test helper, which code is actually under test and what should happen to the different moving parts. Go read the readme in the rspec-given repository, Jim does a way more thorough job at explaining the semantics. Or if you know Cucumber, you can just continue! For the rest, I will be waiting, right here. Back already? Good! Let's continue!

A Primer on RSpec Given

So, if we want to write a Math library that uses a very strange notation, a spec could look like this:
describe "Math" do
	describe "#+" do
		it "adds two values" do
			let(:value_one) { 2 }
			let(:value_two) { 3 }

			Math.+ value_one, value_two

			Math.result.should == 5
		end
	end
end
I know, I know, it's a very contrived example, sorry... :( But what are we testing now? Math.+ or Math.result? With rspec-given, we could specify intent a lot clearer:
describe "Math" do
	describe "#+" do
		context "adds two values" do
			Given(:value_one) {2}
			Given(:value_two) {3}

			When { Math.+ value_one, value_two }

			Then { Math.result.should == 5 }
		end
	end
end
So we have two preconditions (setting value_one and value_two), which are just there to help the test. Then we have the code we actually want to test (Math.+). And we have a post condition, that needs to be true at the end. But in the second example it is clear that we don't test Math.result, but Math.+! Also notice that then replaces the need for an it block. I guess we all can agree that the second example is way more specific which code is there for what purpose. Which I like. But that got me to ask one question: How do you extend the DSL? Let's see:

Extending DSLs

First things first (because it wouldn't make sense to mention the first thing at the end I guess? That saying has never made sense to me, sorry...), I'm looking at rspec-given-2.0.0.beta3. This way, Jim can't ninja change things in there and then tell me I'm totally wrong. Second things second (that doesn't exist, does it?): Let's have a look at the first file: rspec/given.rb. If we ignore RSpec 1 (someones still using that?), we include different files. One of the interesting files is configure.rb. Here the important lines (actually, all lines in this file except the includes):
RSpec.configure do |c|
  c.extend(RSpec::Given::ClassExtensions)
  c.include(RSpec::Given::InstanceExtensions)
  c.include(RSpec::Given::HaveFailed)
end
(On Github) He uses RSpec.configure to extend and include a few extensions. If we take a look at RSpecs rubydocs, we can find RSpec.configure. Even more important, the Configuration object has two methods: extend and include! Now that's handy, because that's exactly what we used! These two are really similar. #extend is used to add methods to example groups (but not the examples itself). It is added to the describe block, but not the it block. Now guess what #include does. Right! It adds methods to the example itself, but not to the example group! Now we know how we can add custom stuff to RSpec examples and example groups! Which is exactly what rspec-given does! The class and instance extensions can be seen in extensions.rb.

That's How You Extend a DSL

RSpec nicely enough provided everything we need to extend it's syntax with our own methods. Of course this isn't just handy for rspec-given. Everyone can extend RSpec with its own syntax, which might come in handy once in a while. I suggest you have a look at the RSpec Configuration object, because it can actually do a lot. Did you know that it can show you the 10 slowest examples, automatically? Or that RSpec has before, after, and around hooks? So not only have we learned what rspec-given is, how it works, but also how we can extend RSpec ourselves! Thanks Jim! If you liked this post, you should consider hiring me freelance, part-time or full-time!

Please Don't Follow Advice From The Experts!

Fun Fact: You most likely fall into the same trap that I do: The expert's dilemma!

The Expert's Dilemma?

Yes, the expert's dilemma. What is it, you might ask. In fact, if you don't ask yourself this question right now, why are you even reading? Maybe this piece then isn't for you. If you are, however, wondering what that is and why you most likely also fall into this trap, read on. The expert's dilemma is quite simple: We try to emulate experts as well as we can. We try to listen to their opinions, follow them, and carry out everything they say. The problem with that? If you are not an expert, most of the things they say don't matter to you! Yes, seriously! Let me illustrate through two examples: Fitness and Starcraft II. I know, I know, both of those have nothing to do with programming (except that progamer looks almost like programmer), but bear with me. Just a moment now.

Let's talk about Starcraft II

Starcraft II is a really simple game. You have so-called harvesters, that gather resources. With these resources you can build more gatherers, units to attack or defend, or new structures and research to make everything stronger. Simple as that. Now of course the pros are talking about balance, and that this unit might be a bit better if it had one attack power more than it has now. Or that if you delay action X, you can do action Y Z seconds faster. And other such fun things. If you follow all this advice, you fall into the trap of the expert's dilemma. When I started playing Starcraft II, it worked something like this: Andreas: "Wohoo, I play Starcraft II. Let's just try to play so I have fun!" The Pro: "But Andreas, what are you doing? You should have a plan, you should scout your opponent!" One week later Andreas: "Whohoo, I play Starcraft II. I even scout!" The Pro: "But Andreas, what are you doing? You should control each unit into his last detail!" One week later Andreas: "Wohoo, I have a lot of free time!" Why did that happen? Because I tried to follow all the experts advice, I forgot that I first need to learn the basics: Gathering resources! So I started to lose a lot and I quit. So all the questions about balance and all that don't play a role for people like me, because other things play a much bigger role. For the pros who play all in the same percentile, they do. But for me?

Let's talk about fitness

Ah, fitness. Where everyone has an opinion. And everyone has a different opinion. It sucks! It makes it hard to start. So let's take the journey for a normal beginner (like me!): Andreas: "Wohoo, I started with a fitness routine! So I run a few times per week and do some bodyweight exercises." The Pro: "But Andreas, what are you doing? Running hurts your muscle gain, so stop that!" One week later Andreas: "Wohoo, I started to do a lot of bodyweight exercises!" The Pro: "But Andreas, what are you doing? If you would vary them more, you could get better results! Also, start doing them on 5 days instead of 3." One week later Andreas: "Wohoo, I'm playing Starcraft II!" Queue a rage comic here... Result: Instead of following what I was doing and enjoying, I stopped doing anything. Instead of getting 80% of the results, I now get 0%!

This Is A Blog About Programming, Right?

Yes, absolutely. Now let me tie it back to programming. TDD, BDD, refactoring, SOLID, performance, method to line ration, line per class ratio, test coverage, newest trends, etc. etc. etc. Sounds familiar? Do you get it now? No? Damn it! That was the point of the whole post. Let me try again: There are a lot of buzzwords and other things to follow, which can choke a beginner. A beginner should be doing. (Careful: Controversial statement following!) If it isn't perfect, who cares? It is more important to do than to do it perfectly!

Experts Are Nice, But...

I know the experts just mean well. But a beginner or intermediate can't possibly follow all advice! It will confuse her or him more than actually following through with it! So don't blindly follow experts advice, just keep being awesome and improving along the way! Don't get bogged down, don't try to do it all perfectly from the beginning. Add stuff slowly and concentrate on the basics. Because like in Starcraft II and in fitness, it is better to do the basics right than butchering all the experts advice. Did you fall into this trap yet? Share your (horror?) stories! If you liked this post, you should consider hiring me freelance, part-time or full-time!

What Is Your Rails Testing Stack?

Developing a Rails application is awesome. Why? Because the testing stack is really, really good! But, this is also a problem. There is so much to choose from, that it sometimes is really hard to figure out what is useful and what isn't. I don't know about you, but my Gemfile tends to have a lot of stuff in the group :development, :test.
group :development, :test do
  gem 'rspec-rails'
  gem 'cucumber-rails', :require => false
  gem 'database_cleaner'
  gem 'capybara'
  gem 'launchy'
  gem 'rr'
  gem 'awesome_print'
  gem 'jasminerice'
  gem 'guard-jasmine'
  gem 'poltergeist'
end
Holy crap that's big! Of course, it also has some helpers only for development:
group :development do
  gem 'guard'
  gem 'libnotify'
  gem 'guard-spork'
  gem 'guard-cucumber'
  gem 'guard-rspec'
  gem 'guard-livereload'
  gem 'spork'
  gem 'pry-rails'
  gem 'pry-stack_explorer'
  gem 'pry-debugger'
end
Not too shaby, huh? So let me get into the details for each of those.

Guard

Guard is an amazing tool to automate nearly everything. Don't believe me? Take a look at all the possible Guard extensions on Github! So what does Guard do? It basically is a system that runs in the background and can watch files. Extensions tell it (through a Guardfile) that it should do something if a specific file pattern changes. For example, the Cucumber guard checks for this:
guard 'cucumber', :cli => "--drb" do
  watch(%r{^features/.+\.feature$})
  watch(%r{^features/support/.+$})          { 'features' }
  watch(%r{^features/step_definitions/(.+)_steps\.rb$}) { |m| Dir[File.join("**/#{m[1]}.feature")][0] || 'features' }
end
It looks at all features in the features directory, and Guard will call the Cucumber guard if one of these files changes. In term, guard-cucumber will call Cucumber and show it to you on the console. Neat, isn't it? Of course, this isn't just for Cucumber! I also use the RSpec guard (for RSpec tests, duh!), Spork and livereload. Spork is a DRb (Distributed Ruby) server that preloads the Rails environment so the tests run faster. Adding this guard makes Guard start the DRb server for Cucumber and RSpec automatically when starting Guard itself. Another very neat guard is guard-livereload. Thanks to an extension in Chrome, this guard will automatically reload the page I have open when I change the ERb or the CSS files. Makes fiddling with CSS a lot easier!

You Have To PRY It From My Cold Dead Hands

PRY is one of those tools that you just have to know (and love). It is an alternative to IRB, and way better! Just add a binding.pry to your code and Pry will open in that context when the code runs over it. It allows you to change into classes and objects to inspect them more closely. And edit the methods if you want to! Some additions I added to Pry (and there are a few Pry plugins) are pry-debugger, so I can step through code and see what it really does. To complement Pry Debugger, I also use pry-stack_explorer. This plugin allows us to move up the stack and see the state of the code there. Of course, in a perfect world, we never have to use one of these tools, but yeah... Real world and these things...

Testing tools

You might now say: "Hey! You said you will talk about testing tools, not support tools!" And you are right! (Though these support tools are amazing for testing!) Yes, testing tools. For my testing needs I use the standard combo RSpec, Cucumber, database_cleaner, Capybara and rr stack. I don't think I have to explain these (Do I? If I do, you should tell me so in the comments!) Additionally I use launchy to launch a browser window with the current state of the page to debug Cucumber steps. I also make copious use of awesome_print, because it is, well, awesome! awesome_print can be used instead of puts or pp to output objects and have an easier time inspecting them. (Pry also can use awesome_print for it's built in printing habits!)

Last But Not Least: JavaScript Testing

Something I struggled with for a bit, but found a good solution for, is JavaScript testing. There are two aspects here: One is Cucumbers Javascript support. For some acceptance tests, I needed Javascript to run. Per default, Cucumber uses Selenium (with a real browser), which is slow and not awesome at all! And then there are the Javascript unit tests, that also need to run. For the unit tests I settled on Jasmine. Why? Because it seems it even tries to be as close to RSpec as possible, so I don't have to learn yet another testing system. To run them, I of course, use a guard: guard-jasmine (Who's surprised?) guard-jasmine uses PhantomJS to run the Javascript tests in a headless mode (without a user interface), which makes them fast (not incredibly fast, but at least fast). Of course we also want to integrate the Jasmine unit tests with the asset pipeline so we can use CoffeeScript. Luckily, Brad Phelan solved that problem and released JasmineRice. Now, this doesn't yet solve the Cucumber problem. For that, we can use Poltergeist. Poltergeist is a PhantomJS driver for Capybara. Do you see how this neatly matches up with guard-jasmine? Nice, isn't it? I just configured Capybara to use Poltergeist instead of Selenium, and now the tests run at an acceptable rate.

So, What's Your stack?

Yes, that is my "standard" Rails testing stack. I think this covers about everything I need. One thing that's not included (because I currently don't need it) is something like VCR. VCR records HTTP interactions, so that during testing the HTTP calls don't have to be made, but can be replayed by VCR. Which makes things even quicker. Have I missed something? Added something superfluous? What do you use for your testing stack? If you liked this post, you should follow me on Twitter and sign up for the RSS feed!

3 Things You Could Know If You Read "Eloquent Ruby" by Russ Olsen: A Review

Starting with Ruby isn't easy. Especially if you are like me. I have, like many before me, a Java background. So I knew statically typed languages, I knew what it meant, I knew what its strength were. And, like the monkey that I was, I thought statically typed languages were the best! Why? IDEs have a really easy time to refactor a lot of code. Everything was clear. Well, you know, the standard arguments you probably also here when we talk about statically typed languages. But this post has nothing to do with statically typed languages, with me having seen the light at the of the tunnel and all that. And since you are reading this, I guess you did the same! But, fun fact, here, for free, no sign-up needed: Ruby can do a lot more. A lot! And it has it warts, and it has its way of doing things. This isn't easy to understand when you are knew, or even when you are experienced. So here are 3 tips on what you could learn from reading Eloquent Ruby by Russ Olsen:

Strings In Ruby Are Awesome! Literally!

Ever had to write a string in Java? I feel sorry for you, because I have written way too many back in the days. Remember "This is a \"String\"" or even better, a Regex? (Hint: "\\..*", and yes, that's basically a smiley. That's also my theory that smilies are a result of Java programmers using Regex). The first string problem is easy to solve in Ruby: Just use a single quote! But even better, if you mix single and double quotes in a sentence ("And here shall "REST" Java. It's done.") we can solve this with a string literal: %q{And here shall "REST" Java. It's done.}. See? No problem. In fact, we could even replace { with '$', since we never use a '$' in our sentence! Or a lot of other special characters for that matter. The Regex (or Regexp?) we could write as /\..*/. No double escaping needed! And yes, it still resembles a smiley, I realize that. It's still a Regex after all! There is a lot more that can be done with string literals, so you should check that out in the book!

Ruby Has Hooks For A Lot Of Things

Ever written a module? Imagine that you are that module. This is carry stuff! Just think about it: You have to blindly stumble around, never knowing which class carries you around! Luckily, Ruby offers us hooks that we can use to alleviate these problems. Each module has a self.included method that you can use. It is called every time a class, well, includes this module. Even better, it even tells you which class it was! This way you can build up a list of all classes that have this module included. Doesn't sound very handy? But maybe, when the module is included, you also want to extend that class. Maybe you have a method that should not be on the class, but on the object! With self.included and other hooks, everything's possible!

How To Build Your Own DSL

DSLs. How much we all love these little things, and how much we use them each and every day. Thank you, dynamic typing! Probably you have at least seen some RSpec code. Yes, all that let, describe, and other syntactic sugar is just a DSL, implemented in Ruby. Do you want to know how to do this? Then go read the book! It has a few caveats that I can't all list here, because this is just a blog, not a book! Even better, he also mentions how to build an external DSL. An external DSL is not written in Ruby, and has to be parsed "by hand" with Ruby (without the Ruby interpreter helping). Which does have its pros and cons, as you might imagine.

Conclusion

All in all, a very good book that opened my eyes (and still does, on my second reading of it!). One of the really nice features about the book is its structure. Russ gives us first an introduction why we want to do this, then tells us how. So far so good. But then he continues: He also tells us how you can get in trouble with it. To top it off, he shows some examples of the described concept in the wild. If you are a Rubyist and you haven't yet read Eloquent Ruby, I highly suggest you do. Because Ruby can do a lot more than you might image! If you liked this post, you should follow me on Twitter and sign up for the RSS feed!

What Others Know About Compression That You Don't

Ah, the compression formats. Always a mystery. Always just something we will use, but something we never understand. For one thing, this makes sense, considering Wikipedia alone lists 15 different compression formats. There are the well-known bz2, gz and LZMA algorithms. And the less well-known rz, ?Q? and ??_. Now that last one makes me happy this isn't a podcast... Though for this post we will concentrate on Deflate. Deflate is not only used in gzip and zip, but is also a fan favorite to compress web assets when sent to the browser.

The Basic Building Blocks

Underneath Deflate uses LZ77 combined with a Huffman encoding. LZ77 is used to detect duplicate strings and insert a so-called back reference (more on this later). Huffman Encoding is similar to ASCII, but instead of always using 8 bits, it stores the most used characters in 3 bits, and the least used ones, with, well, more bits. More on this also later.

Come On, Deflate Already!

First, Deflate chops up all parts into blocks to make processing easier. Then it decides in which mode it should encode that block:
  1. Store raw stream
  2. Store with a static Huffman encoding
  3. Store with a dynamic Huffman Encoding
Now, it still doesn't deflate. But lets see. First mode is raw mode. In raw mode, Deflate decides that the data is already compressed and it can't do a better job. So it just inserts its header to say "I can't do anything with that, this is already compressed, dummy!". That's why, the next time a coworker shouts at you why you didn't ZIP these files, you can shout back: "D'oh, mode 1, man!". Second mode is static Huffman mode. This just means that Deflate decides that the data is standard enough that the standard Huffman Encoding table should be used. For example useful for English text. This way Deflate doesn't have to add a table as overhead to the compressed content. Third is dynamic Huffman mode. With dynamic Huffman mode, the content gets compressed with a Huffman Encoding, but the encoding table is generated based on the input. So for example, in Finish, the shortest code would probably be "รถ", or something like that. In @zedshaw's recent tweets it could be "C", "F", "K" and "U", who knows! But that leaves us with two question: Huffman Encoding, wut? And LZ77?

Huffman Encoding

Huffman Encoding is something that a guy called Huffman developed while getting a Ph.D from MTI MIT. Nothing surprising until now! The paper on it is actually from 1952. Damn it! That's double my age! That paper should feel old! What it basically says: Tally up all individual characters. The most used one gets the shortest code (at 3 bits). Then continue down until you arrived at the end. Actually, Wikipedia does a better job at that than I ever could, so here's a picture: [caption id="attachment_241" align="alignnone" width="600"]Huffman Coding Tree Huffman Coding Tree[/caption] (Source: http://en.wikipedia.org/wiki/File:Huffman_tree_2.svg) So that one sounded scary, turned out to be really easy!

So What About That LZ77

The next part in this tour around compression algorithms leads us right back into history. Can't we get something new going in the compression world? LZ77 was introduced in a paper in, well, 1977 by Lempel and Ziv. LZ77 is a so-called sliding window compressor. At least they came up with impressive names back then. Why sliding window? It checks the last 2, 4 or 32 kB for repeating strings. Simple as that. Should it find one such repeating string, instead of adding that string to the output stream, it puts a back marker in that basically says: If you want to finish reading that sentence, go back to page 32, line 17, and read from character 17 to character 34. That lazy bastard!

Now I See How They Tie Together

Well I hope you do! Else I failed and you should shame in the comments. :) Or of course I can try again: First, Deflate splits up the document into blocks to make working with it easier. Then (lets assume the input isn't already compressed) it runs each block through LZ77 to get all the obvious double strings out of the way. Then it sends it through a Huffman Encoding to make it even shorter. So yes, that's deflate.

What About The Others (BZIP2 Anyone?)

Not all of them use this simple approach, because simple apparently isn't good enough. BZIP2 goes through 9 different stages.
  1. Replacing all 4 or more repeating characters with just 4 characters. (There you see, a geek had to make this algorithm for comic books! "AAAAAAAAAA!" is so much longer than "AAA\7!") So called Comic Book Encoding... Unfortunately not, it's simply called Run Length Encoding. How boring!
  2. Next they wheel around the characters. Which basically means a fancy way to sort the same letters after each other so compression is easier. The so called Burrows-Wheeler Transform.
  3. Replace all repeating characters with 0, so we have as many 0 as possible. Could also be any other character. The Move-to-Front Transform (Yes, I hate these people too at concerts).
  4. And again, after sorting everything possible, we do a Run-length Encoding.
  5. Then a Huffman Encoding. Don't tell me now I have to explain what that is!
  6. Then again we see Huffmaaa\7n! This time with multiple tables per block to make the output even shorter.
  7. 8. and 9. Add some binary trickery.
So yes, you can use a lot of other things, but it all comes back to making the output even shorter. Surprising, hu?

So That's Compression

Yes, that's compression. Use a bit of special encoding, remove duplicates, and you are halfway there. Of course, this is just for Deflate. We saw that BZIP2 is already a lot more complicated and uses a lot of different tricks to make the output even smaller, but then again, it is also slower! If you liked this post, you should follow me on Twitter and sign up for the RSS feed! B58P3MMWCDTH