Ruby, etc

Learn all the things

He who enjoys doing and enjoys what he has done is happy. - Fortune Cookie

Tagging in Robot Framework

I talked about the awesome robot framework in the previous post and I wanted to talk about Tagging since I think this is very useful.

Tagging is one way to group tests so you run a subset of tests. In your test case, add this:

1
2
3
4
5
Homepage Loads
  [Tags]  smoke
  Open Homepage
  Element Should Be Visible  ${FIND_LOGO}
  [Teardown]  Close Browser

Results to run just one tag:

1
2
3
4
5
6
7
8
9
10
11
12
▶ robot --include smoke youtube.robot
==============================================================================
Youtube :: A test to demo testing YouTube
==============================================================================
Homepage Loads                                                        | PASS |
------------------------------------------------------------------------------
Search Loads Results                                                  | PASS |
------------------------------------------------------------------------------
Youtube :: A test to demo testing YouTube                             | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================

You can create a focus so it is easy to run that one test as you are developing it. I have used a similar technique when working on rspec tests. When the test works as you wanted, remove the focus tag. :)

When you look at the report you can see the tags assigned to teach tests. This report was run on all tests. Screenshot

Automation Testing With Robot Framework

On one of my projects recently, I was tasked with finding an automation framework. Looking at my options..

  • Cucumber (ruby, although other languages could be used)
  • Capybara / Rspec (ruby)
  • Robot Framework (python)

All of the options for web testing use either Selenium or Watir. I have only did a little with Watir, so I focused on something with Selenium for my first pass of experimentation.

I love testing if you know anything about me, I’m nuts about testing. Front End Testing however has been fruitless IMHO .. until a few years ago I discovered page objects and it became tolerable to do frontend testing. My projects changed and I didn’t really do much for a few years with web frontend testing.

Recently I have been looking into Python and discovered Robot Framework, meant to test a variety of things in addition to the web. The Selenium2 package is especially nice.

You can have a robot script as a text field, in a TSV, or other types of strutured text. I find the text file works for me.

Here we can define a Test Case:

1
2
3
4
Homepage Loads
  Open Browser  https://www.youtube.com  Firefox
  Element Should Be Visible  id=yt-masthead-logo-fragment 
  [Teardown]  Close Browser

This script opens the browser to youtube using firefox. Then checks that element located by id=yt-masthead-logo-fragment is visible (ie on the page, not display: none).

you must make sure to have two spaces between the elements

Open Browserhttps://www.youtube.comFirefox

Or it won’t be able to parse. I guess this is when having it in a spreadsheet (TSV) would come in handy. But it’s not too bad to get the hang of. Plus if you use nice editors like PyCharm and the Robot Framework Plugin, it has nice syntax highlighting for you.

Running the test case like this:

1
robot youtube.robot

And results look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
▶ robot youtube.robot
==============================================================================
Youtube :: A test to demo testing YouTube
==============================================================================
Homepage Loads                                                        | PASS |
------------------------------------------------------------------------------
Youtube :: A test to demo testing YouTube                             | PASS |
1 critical tests, 1 passed, 0 failed
1 tests total, 1 passed, 0 failed
==============================================================================
Output:  robot-demo/output.xml
Log:     robot-demo/log.html
Report:  robot-demo/report.html

`

There that is just a taste of what it can do. If you want to see what all you need installed you can see my robot demo and some more complex tests. More testing posts to come on this, I think this is a very excellent framework … even if it is not ruby :)

My Favorite Tech Podcasts

I saw someone asking about tech podcasts recently and I named off a few, but I think it is easier to put it in a blog post :) Ordered in roughly the order I am into at the moment :)

General Tech

  • The Changelog - open source discussions, interviews with a wide varity of languages
  • DotNetRocks - You might think it is all about .NET but it is not, they cover a wide variety of topics.
  • Functional Geekery - Wide variety of topics on Functional Languages.

Testing

Python

Clojure

  • defn podcast - The hosts are hillarious on this podcast and I enjoy the banter, mostly interviews with clojure developers
  • Cognicast - Hosts are from Cognitect, the “stewards of the clojure language”.

Elixir

Ruby

  • Ruby Rogues - I listen sometimes, I generally browse the topics to see if it looks interesting.

Game Development

Misc

  • Two Keto Dudes - About the Keto way of eating. BACON BACON!!! Also I am a bacontarian. :)
  • Into The Nexus - About Blizzard’s game Heroes of the Storm

Using Jshint

Javascript. It’s a language that some of us love to hate, me occasionally. Here’s a few notes to myself to make JS a little better :)

Use these settings in JSHint to avoid instanity:

1
2
3
4
file: .jshintrc
{
  "flagname" : true
}

If you have settings that apply to only your test environment you can put those in test/.jshintrc (maybe allowing global var for your test function) .. you can extend a confiuration in another file with this option `”extends” : “../.jshintrc”

Sometimes you just can’t please everyone, including the linter. Here’s how to ignore blocks or lines from the linter:

To ignore a block of code use

1
2
3
/* jshint ignore:start */
myBadJsHere
/* jshint ignore:end */

to just ignore a single line:

1
myBadJsHere // jshint ignore:line

Obviously you shouldn’t let the linter ignore code but you may have a good reason to do so.

Handy thing to have in your .jshintrc file:

eqeqeq

1 != “1” right? comparing a number to a string? no, JS says sure its the same using automatic type conversion. Use this flag to warn you and remind you to use triple equals to not do automagic type conversion.

Switch statements

This was just a neat thing I learned today, in JS switch/case statements you need to do a return or break or you will fall through to the next option. Leave a note for the next guy that says you did mean to fall through with a comment /* falls through */ and you will silence the warning from jslint. Kinda cool, huh?

Use Strict

"use strict";

This tells javascript to cut the funny business and not be so helpful and error when stuff should error.

I picked up some of these tips watching Pluralsight: JavaScript Best Practices

Thanks for humoring me as I write these notes to myself :)

Ruby Like Python

I did python.. awhile back, it was like ruby in some ways and in some ways not. So I picked one, I went full blown into Ruby. Recently I’ve seen lots of job postings for Python so I figured I’d give it another go :)

Slicing

In Python:

1
2
3
4
5
6
7
>>> message = "hello world"

>>> message[0:5]
'hello'

>>> message[6:]
'world'

Then in ruby

1
2
3
4
5
6
7
8
2.4.0 :001 > message = "hello world"
 => "hello world"

2.4.0 :002 > message[0..5]
 => "hello "

2.4.0 :003 > message[6..-1]
 => "world"

Eveything in ruby is an object and objects have methods. String has a method slice, aliased to [] (cool!) and the .. is a range. You could use a comma but there is so way to tell it to use the end unless using a range. If you look at the params for the slice method it is a fixnum or fixnum, fixnum or a range. It can also take a string (if you use the method name) but that is a strange use.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
2.4.0 :005 > message
 => "hello world"

2.4.0 :006 > message.slice("ello")
 => "ello"

2.4.0 :007 > message
 => "hello world"

2.4.0 :008 > message.slice!("ello")
 => "ello"

2.4.0 :009 > message
 => "h world"

Unless you use slice! (bang) does it actually change/do anything remotely interesting. Probably I would find a more clear way to do it than using slice with a string.

Well this is just one aspect of the differences and I am not sure if there are differences in the versions of python too.

Setting Up a New Ruby or Rails Project

A checklist to follow when starting a new project if you are using rbenv or rvm:

Make sure this is in your .bashrc or .zshrc

1
alias be="bundle exec"

Then any time you would need bundle exec you can just use be. Or alternatively, Hal Fulton pointed out you can do bundle exec bash to get a bash shell that would be the same as using be each time :)

  1. Create a directory for your project and change into it.
  2. Create a Ruby version file: echo "2.4.0" > .ruby_version.
  3. Change out of directory and back in, and ensure your version of ruby is correct with ruby -v.
  4. Make sure Bundler is installed: gem install bundler.
  5. Create a Gemfile bundle init.
  6. Setup git repo with git init.
  7. Create a README file and put the name of your project and what it is used for.
  8. Add all files git add ..
  9. Commit git commit -m "inital commit".

Then if you are making a ruby gem:

  1. Use bundle gem myawesomegemor do it by hand (helps you to remember.. hehe)
  2. Create a gemspec touch myawesomelibrary.gemspec.
  3. Make directory: mkdir lib.
  4. Make library file: touch lib/myawesomelibrary.rb.
  5. Make test directory: mkdir test.
  6. Make test file: touch test/myawesomelibrary_test.rb.

Gem spec template:

1
2
3
4
5
6
7
8
9
10
11
12
13
Gem::Specification.new do |s|
  s.name        = 'myawesomelibrary'
  s.version     = '0.0.0'
  s.date        = '2017-01-01'
  s.summary     = "My awesome library summary"
  s.description = "My awesome library description"
  s.authors     = ["Awesome Programmer"]
  s.email       = 'myself@awesomeprogramming.com'
  s.files       = ["lib/myawesomelibrary.rb"]
  s.homepage    =
    'http://rubygems.org/gems/myawesomelibrary'
  s.license       = 'MIT'
end

If you are making a rails app:

Add the version of Rails you want to use to the Gemfile ie: rails "5.0.1" and bundle install. Then do rails new . to create a rails app with the same name as current directory.

Records in Clojure

I haven’t used records before in a project and when reading some code that used it, realize I really need to learn about records. It’s not complicated and actually kind of cool! Here’s my experimentation.

To start off, I re-read the first chapter in ClojureApplied where it talks about Modeling your Entities.

I took a familar problem, my recipe-api. This is a recipe api that uses liberator api reading recipes from a database. But, I can use the concept here.

1
2
user=> (defrecord Recipe [name link source])
user.Recipe

When you create a record, you get two factory functions for free to create instances of it. The first, a positional function:

1
2
user=> (def tacos (->Recipe "Tacos" "www.tacorecipes.com" "mom"))
#'user/tacos

You must put values for all the fields in or you get

1
2
3
user=> (def tacos (->Recipe "Tacos" "www.tacorecipes.com"))

CompilerException clojure.lang.ArityException: Wrong number of args (2) passed to: user/eval10000/->Recipe--10017, compiling:(form-init4856550892047924668.clj:1:12)

The other way you can create an instance of a record is using a map.

1
2
3
4
user=> (def pizza (map->Recipe {:name "Pizza" :source "dad"}))
#'user/pizza
user=> pizza
#user.Recipe{:name "Pizza", :link nil, :source "dad"}

We didn’t have a value for link in the map so it still let us create the instance and automatically assigned the value of nil. This might be a reason to use the map-> version over the positional version, if you don’t know all the values at that time. They work like maps, you can add in missing values later with assoc:

1
2
user=> (assoc pizza :link "www.pizzarecipes.com")
#user.Recipe{:name "Pizza", :link "www.pizzarecipes.com", :source "dad"}

Accessing the the values in a record is just like a map:

1
2
3
4
user=> (:name tacos)
"Tacos"
user=> (:source tacos)
"mom"

The chapter has a great section on why you would want to choose Records over Maps you should read! I won’t spoil the fun here, but there are some compelling reasons when to use records over a map.

Two Cool Tools for Clojure Development

I’ve recently used a couple tools that I want to write about because I think they are pretty useful.

1. Eastwood

Eastwood is a linter, which will check your syntax to see if it is valid. BTW I have actually made this mistake for real! The github page lists all the things it checks for and a lot of how to configure eastwood.

To get started, Install into your lein profiles.clj:

1
{:user {:plugins [[jonase/eastwood "0.2.1"]]}}

My function, which by the way, is awesome:

1
2
3
(defn my-awesome-function [x y z]
"My function is awesome"
(+ x y z))

Running lein eastwood in the project shows me:

1
2
3
4
5
6
7
8
9
10
11
lein eastwood
== Eastwood 0.2.1 Clojure 1.7.0 JVM 1.8.0_60
Directories scanned for source files:
src test
== Linting blog-post.core ==
Entering directory `/Users/nolastowe/practicing/clojure/blog-post'
src/blog_post/core.clj:5:7: misplaced-docstrings: Possibly misplaced docstring, my-awesome-function
src/blog_post/core.clj:5:1: unused-ret-vals: Constant value is discarded: "My function is awesome"
== Linting blog-post.core-test ==
== Warnings: 2 (not including reflection warnings)  Exceptions thrown: 0
Subprocess failed

Doh. The docstring is in the wrong place, correcting that and eastwood is happy. That is an easy mistake to make!

1
2
3
4
5
6
7
lein eastwood
== Eastwood 0.2.1 Clojure 1.7.0 JVM 1.8.0_60
Directories scanned for source files:
src test
== Linting blog-post.core ==
== Linting blog-post.core-test ==
== Warnings: 0 (not including reflection warnings)  Exceptions thrown: 0

I’ve used eastwood a lot when I was developing amazon lambda functions, when I couldn’t see the result till I’ve uploaded to amazon. At first was frustrated only after all that to find an error which required me to do another compile to jar and upload. Now I use it before commit or when i’ve completed a unit of work or when I just can’t figure why something is not working..always a chance I did something stupid with syntax.

2. Criterium

This library is for benchmarking clojure and I was using it to compare different implementations when I was working on adding some string functions to Clojure. Initially I added it to my lein profiles (similar to how i did with eastwood) which works fine for testing in the repl. Ghadi saw me talking about using it in Clojurians Slack and gave me a bash script to start any version of clojure and criterium with a repl:

1
2
3
4
5
6
#!/bin/bash
M2=$HOME/.m2/repository
CLASSPATH=${M2}/criterium/criterium/0.4.3/criterium-0.4.3.jar:src
JOPTS='-Xmx4G -XX:+UseG1GC'
CLOJURE_JAR=$M2/org/clojure/clojure/${CLOJURE_VERSION}/clojure-${CLOJURE_VERSION}.jar
java $JOPTS -cp ${CLASSPATH}:${CLOJURE_JAR} clojure.main

Then save it as repl-version.sh and run with CLOJURE_VERSION=1.7.0 repl-version.sh to open a repl, then you can test away. I see this being useful when you are working on clojure itself or want to compare things in different versions of clojure. I used it when I was working on a patch for Clojure.

One of the things I used it for was figuring wanted to compare char? and instance? which should I use.. this is a true story! I was trying to figure this out one day..

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
user=> (quick-bench (char? \c))
Evaluation count : 39012534 in 6 samples of 6502089 calls.
Execution time mean : 5.188821 ns
Execution time std-deviation : 0.351751 ns
Execution time lower quantile : 4.755146 ns ( 2.5%)
Execution time upper quantile : 5.544008 ns (97.5%)
Overhead used : 10.020671 ns

user=> (quick-bench (instance? Character  \c))
Evaluation count : 51496848 in 6 samples of 8582808 calls.
Execution time mean : 2.510016 ns
Execution time std-deviation : 0.701679 ns
Execution time lower quantile : 1.693617 ns ( 2.5%)
Execution time upper quantile : 3.375066 ns (97.5%)
Overhead used : 9.812462 ns
nil

I wondered, how does char? work? So I looked at the source:

1
2
3
4
5
6
7
8
user=> (source char?)
(def
^{:arglists '([x])
:doc "Return true if x is a Character"
:added "1.0"
:static true}
char? (fn ^:static char? [x] (instance? Character x)))
nil

Ahh, I see, it just checked the instance? .. so I look what that function does:

1
2
3
4
5
6
7
8
user=> (source instance?)
(def
^{:arglists '([^Class c x])
:doc "Evaluates x and tests if it is an instance of the class
c. Returns true or false"
:added "1.0"}
instance? (fn instance? [^Class c x] (. c (isInstance x))))
nil

So of course, instance will be faster than char …. If you know your class of object, it looks like instance? will always be a little faster. But there ya have benchmarking in Clojure with Criterium. :)

When reviewing this post, Ghadi said:

One small note, instance? is intrinsified by the compiler. It turns into a bytecode instruction. Which you can see in this clojure source. It’s faster than a function call.

Book Review - Clojure Applied

Written by Alex Miller and Ben Vangrift published by Pragmatic Programmers.

I am a junkie for books and Clojure books are no exception. I have been doing Clojure since Jan 2014 and feel like I kinda got the hang of it but now what? When I saw the Clojure Applied “From Practice to Practitioner” was being written I kept a close eye on Pragmatic Programmer’s coming soon list and an eye on twitter. The day I heard the beta book was available I bought it. When I was offered a free copy to do this review, I said thanks but I already have it and I’ll still be happy to write a review. :)

I was not disappointed as I dived into the book, this is just the thing I needed. Here’s is how to do clojure for the real world with real application of clojure’s concepts applied.

This book is in 3 parts:

  • Foundations
  • Applications
  • Practices

Foundations

Chapter 1: Modeling your Domain

This section covers aspects of your application including modeling your domain using maps and record and when to use each of those. Getting your mind around immutable data is one of the hardest parts for me in getting into clojure and the discussion on entities here really helped solidify the concepts. Once your data is modeled you want to do operations, in OOP languages you have polymorphism. In clojure there are multi methods and protocols. Here are examples of each and when you would use them.

Chapter 2: Collecting and organizing your data.

This chapter talks about collections and which one to use and how to write a custom sorting function. Then it goes on to talk about updating data - wait I thought data was immutable in clojure? It is, so when you update a collection it returns a new copy with the data updated. Its typical of mutable functions to end in ! (danger Will Robinson!), This chapter closes with how to define a “new” datatype and what is involved there.

Chapter 3: Processing Sequential Data

This chapter starts with explaining how sequences are an abstraction connecting collections and the functions that act on those collection. This is the really beauty of clojure and at first mind blowing then once it sinks in, you can imagine how you could have thought differently about the concept. Reducing, Filtering, Selection, Sorting, Removing Duplicates and grouping are all operations you will want to do on your collection once you have it.

Applications

Chapter 4: State, Identity, and Change

Change, if our applications couldn’t change they would be boring, probably even useless. If you need more than one change at a time, you should use a transaction. Different data types (atom, ref, etc) have different functions to update and this goes over each of them in detail.

Chapter 5: Use your cores

This chapter talks about threads and introduces the concept of agents and promises to transfer a value from one state to the other. The talk about using reducers to process data in parallel and the performance considerations. Ending with with discussion about thinking in process using channels in core.async to create a sort of a pipeline for processing data. More meat to this chapter than I can summarize here.

Chapter 6: Creating Components

One of my favorite chapters. Now that you have your structure and processing down, how to combine it into something we can put in a package with a nice bow? There are a few ways to organize this and the first is a namespace. You want to use one or more for your code so it is easily read and comprehended. The chapter goes over some common namespaces used in project and what might go into those. You can connect your components with channels and structure their lifecycle with a record and functions to make start and stop. Leading to the next chapter…

Chapter 7: Compose your Application

Composing your application! Early in the chapter it starts talking about Stuart Sierra’s library name precisely that. Component. It’s like the previous chapter laid the foundation of why, and now this is how. Then putting those into a system and using profiles and configs to make your code operate in different environments (dev, staging etc).

Practice

Chapter 8: Testing Clojure

Ok maybe this is my favorite chapter. My love of testing has been equated with “sickness” for loving tests so much. This chapter kicks off with repl-based testing, by developing your function and calling it in the repl to work out the inputs/outputs. Then moving on to example based testing which I know as unit testing. The new part to me is in Clojure are “are” test function which can easily test a lot of short tests. Testing exceptions is also a thing which is mentioned briefly. Then some other features of clojure test are fixtures, a handy way to provide baseline data for testing. The section ends with a mention of ‘Expections’ test library which takes a different approach than ‘clojure.test’. This chapter ends with talking about property based testing and generators, which if you don’t know about them will blow your mind and leaving you wondering why we haven’t been testing this way all along, :)

Chapter 9: Formatting Data

This chapter seems shorter compared to the other chapters, talking about edn, son and transit. The first two I get and using them in clojure is awesome. But Transit, the authors follow the explanation with a concrete example, I think I get it but I will need to use it in a project for it to really sink in.

Chapter 10: Getting out the Door

This is where it talks about publishing your code and deployment. A few things to think about when publishing your code is to choose the collaboration platform (Github, Bitbucket etc), the contributing agreements and licensing and the minimum files you should have in your repo. When your code is ready to be published as a library most people use Clojars, Maven Central Repository or a self hosted Maven repo. The last part of this chapter is deploying your application to heroku or provisioning your own server by running a jar or to deploying to an application server.

Appendix

This contains two sections: Roots which talks about some of the concepts and Thinking in Clojure which describes the mindset of how clojure is written and what I think community strives to be like.

Overall this is a fantastic book and I’ve tried to summarize the key points to help you make the decision if this book would help you in your clojure career. I bet it will! :)

Enumerable: Any?

This is about the handy any? method in Enumerable

The documentation says

Passes each element of the collection to the given block. The method returns true if the block ever returns a value other than false or nil. If the block is not given, Ruby adds an implicit block of { |obj| obj } that will cause any? to return true if at least one of the collection members is not false or nil.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
class Taco
  attr_accessor :meat

  def initialize(meat)
    @meat = meat
  end

  def beef?
    @meat == :beef
  end

  def chicken?
    @meat == :chicken
  end

end

Lets test it out

1
2
3
4
5
6
7
8
irb(main):018:0> taco = Taco.new(:beef)
=> #<Taco:0x007fa374070278 @meat=:beef>

irb(main)> taco.beef?
=> true

irb(main)> taco.chicken?
=> false

Lets create two instances

1
2
3
tacos = []
tacos.push(Taco.new(:beef))
tacos.push(Taco.new(:chicken))

Now since the array has enumerable and we made some handy methods to test the meat of our tacos we can easily check our array to see if we have any chicken tacos:

1
tacos.any? { |taco| taco.chicken? }

If all the elements of the array are booleans as in this case we don’t need the block version:

1
2
3
4
5
irb(main)> possible_chicken_tacos = [tacos[0].chicken?, tacos[1].chicken?]
=> [true, false]

irb(main)> possible_chicken_tacos.any?
=> true

Yep, today was definately a chicken taco day. :)