CircleCI

My CI tool that I use with the team at everquote

Screen Shot 2015-11-24 at 10.18.47 AM

Advertisements

Basic Selenium Test

A basic template for reference

# filename: search.rb

require "json"
require "selenium-webdriver"
require "rspec"
include RSpec::Expectations

describe "Search" do

  before(:each) do
    @driver = Selenium::WebDriver.for :firefox
    @base_url = "https://www.google.com/"
    @accept_next_alert = true
    @driver.manage.timeouts.implicit_wait = 30
    @verification_errors = []
  end

  after(:each) do
    @driver.quit
    @verification_errors.should == []
  end

  it "test_search" do
    @driver.get(@base_url + "/?gws_rd=ssl")
    @driver.find_element(:id, "lst-ib").clear
    @driver.find_element(:id, "lst-ib").send_keys "elemental selenium tips"
    # ERROR: Caught exception [ERROR: Unsupported command [selectWindow | null | ]]
    (@driver.find_element(:link, "Tip Archive - Elemental Selenium").text).should == "Tip Archive - Elemental Selenium"
  end

  def element_present?(how, what)
    ${receiver}.find_element(how, what)
    true
  rescue Selenium::WebDriver::Error::NoSuchElementError
    false
  end

  def alert_present?()
    ${receiver}.switch_to.alert
    true
  rescue Selenium::WebDriver::Error::NoAlertPresentError
    false
  end

  def verify(&blk)
    yield
  rescue ExpectationNotMetError => ex
    @verification_errors << ex
  end

  def close_alert_and_get_its_text(how, what)
    alert = ${receiver}.switch_to().alert()
    alert_text = alert.text
    if (@accept_next_alert) then
      alert.accept()
    else
      alert.dismiss()
    end
    alert_text
  ensure
    @accept_next_alert = true
  end
end

Good Ideas

I’m going to use this post to collect good ideas.

I’m calling it “good ideas” instead of ‘best practices’, ‘rules’, guidelines’, etc as they all other other connotations
The idea is to have a good idea… and then real world examples of it “in action”.

1. Small commits are good
Examples:
A – 09/21/2015 – everquote
I broke up some simple changes into two commits as the changes were unrelated.
Then CI broke due to one of the file changes
As that file change was in its own commit I was able to revert that commit on its own and continue.

2. One assertion per test
Example:
Error text should say “must be numeric” in red.
1. Check the text
2. Check the color of test text.
This one if only one of things is not true the error message will be more specific.

sed and awk. my new best friends and I are going for longer walks

Here’s our latest trip!

$ cat sed_one_test_case_html_to_ruby_functions.sh
sed_test_case_html_to_ruby() {
file_to_convert="$1"
header_number_of_lines="14"
printf "# File: $file_to_convert\n"
cat $file_to_convert | tail +$header_number_of_lines |
sed -E '
/<td><\/td>/d
/<\/tr>/d
s/(<tr>|<td>)//
s/<\/td>//
s/<\/tbody><\/table>//
s/.*<thead>//
s/.*<\/thead><tbody>//
s/(<\/html>|<\/body>)//
s/.*~.*//
s/\${//g
s/\}//
s/\}$//
s/<\!--/#/
s/#<<<<</<<<<</
s/#>>>>>/>>>>>/
s/-->$//
s/[[:space:]]//
s/^type$/fill_in/
s/# #/#/
s/^##/# /' |
awk '{if($0=="click" || $0=="submitAndWait" || $0=="check" || $0=="open" || $0=="waitForElementPresent" || $0=="assertElementPresent" || $0=="waitForVisible" || $0=="clickAndWait" || $0=="pause" || $0=="submit" || $0 =="assertVisible")
{getline n;printf "%s(%s)\n", $0, n}else{print}}' |
awk '{if($0=="fill_in" || $0=="select"){getline n;getline n2; printf "%s(%s, %s)\n", $0, n, n2}else{print}}' |
sed '
/open(/ {
s/open(/visit("http:\/\/staging.everquote\//
s/)/")/
}' |
sed '/^$/d' |
sed $'
/^waitForVisible/ {
s/waitForVisible/expect/
s/)$/).to be_visible\\\n/
}
/^waitForElementPresent/ {
s/waitForElementPresent/expect/
s/)$/).to be_present\\\n/
}
s/^clickAndWait/click/
s/^submitAndWait/submit/
s/^storeConfirmation/close_alert_and_get_its_text/' |
sed '/css=/  {
s/css=/&"/
s/)/")/
}
/link=/  {
s/link=/&"/
s/)/")/
}
/(\/\// {
s/\/\//"\/\//
s/)$/")/
}
/fill_in(/ {
s/, )/, "")/
s/, /, with: /
s/, with: #/, with: "") #/
}' |
awk '{if ($1 ~ /^select\(/ ) {printf "SELECT(%s %s\n", $2, $1} else{print}}' |
sed '/^SELECT/ {
s/) select(/,/
s/,$/)/
s/,/, from: /
s/SELECT/select/
}
/click(/ {
s/click(/find(/
s/)/).click/
}' |
awk '{if ($1 ~ /^>>>>>/ ) {printf "it \"";print $0"\" do"} else{print}}' |
sed $'/it/ {
s/it " /it "/
s/>>>>>//
s/Begin //
}' |
awk '{if ($1 ~ /^<<<<</ ) {printf "  end";print ""} else{print}}' |
awk -F"(" '{if ($1 ~ /^assertVisible/ ) {printf "expect(%s.to be_visible\n",$2;print ""} else{print}}' |
awk '!/^it/ && !/^end/ {printf("  %s\n", $0);next};1' |
awk '!/^before/ && !/^end/ {printf("  %s\n", $0);next};1' |
awk -F"(" '{if ($1 ~ /    expect/ ) {printf "    expect(";printf $2"\n"} else {print}}' |
awk -F"(" '{if ($1 ~ /^    end$/ ) {printf "  end\n    it \"next step\" do\n"} else {print}}' |
awk 'BEGIN {print "scenario \"'"$file_to_convert"'\" do"} {print} END {printf "  end\nend\n"}' |
sed '
/^$/d
/^      end$/d
s/select(, from:  select(css_vehicle1_auto_make/select(auto_make_1, from: css_vehicle1_auto_make)/' |
# Above s/select... is a brute force hack to fix one line I can't figure out with sed or awk
sed $'
s/select(auto_make_1, from: css_vehicle1_auto_make)/with_ajax_wait do\\\n      &\\\n    end\\\n/
s/select(auto_make_2, from: css_vehicle1_auto_make)/with_ajax_wait do\\\n      &\\\n    end\\\n/
s/select(auto_model_1, from: css_vehicle1_auto_model)/with_ajax_wait do\\\n      &\\\n    end\\\n/
s/select(auto_submodel, from: css_vehicle1_auto_submodel)/with_ajax_wait do\\\n      &\\\n    end\\\n/
s/select(auto_make_1, from: css_vehicle_make)/with_ajax_wait do\\\n      &\\\n    end\\\n/
s/select(auto_make_2, from: css_vehicle_make)/with_ajax_wait do\\\n      &\\\n    end\\\n/
s/select(auto_model_1, from: css_vehicle_model)/with_ajax_wait do\\\n      &\\\n    end\\\n/
s/select(auto_submodel, from: css_vehicle_submodel)/with_ajax_wait do\\\n      &\\\n    end\\\n/' |
awk '{if (NR != 2) {print}}' |
sed $'
s/^[[:space:]]*it/  end\\\n  it/
/^[[:space:]]*pause(/d
/^[[:space:]]*$/d
/expect(.*).*to be_visible/ {
s/expect(/with_ajax_wait do\\\n      &/
s/be_visible/&\\\n    end/
}'
}

Identifying Web Elements for UI Automation

Identifying web UI elements at everquote in rails based applications is interesting.

The following facts and approaches have emerged:

– Our forms flip between new and edit.  One of the consequences is that we have to force 1 required field to be empty for SAD flows to work correctly.
– All database backed fields should use css NAMES for identification instead of ID.  This is because database backed elements in rails automatically generate the NAME attributes and thus are automatically consistent regardless of placement and regardless of programmer preference or style
– The mixed `css=` and `name=` are best managed by just using `css=` in the Selenium IDE.
– set speed and deleteAllVisible cookies steps didn’t seem to really help.
– for select dropdowns, waitforElementPresent and select should use the same format for identifying the select
– Most of the “action” buttons and links should have human readable characteristics, e.g. `css=a.add_vehicle` is clearly a link for adding a vehicle.  Similarly `css=a.remove-violation[data-method=delete]` is clearly a “delete violation” link.
– delete buttons should be selected with `data-method=delete` when that attribute it is available on the element.
– selection should generally not be based on layout tags such as `div`, `td`, `span`, etc.  `li` is somewhat of an exception as lists of items are a generic data form. Exceptions to the above are used for scoping if nothing else currently exist.

Example:

Screen Shot 2015-08-10 at 8.40.53 AM

Introducing QA in an Agile environment

Useful for references for my on-going work at everquote.com

http://marker.to/VsBhbB

http://marker.to/4JYac9

http://www.istqb.org/downloads/finish/52/138.html
Whole Team Approach – Page 10
Scrum/Kanban – Page 12
Retrospectives – Page 14
CI – Page 14
Iteration planning – Page 16
Organizational Options for Independent Testing – Page 22
Communicating Test Status, Progress, and Product Quality – Page 23
The Role of a Tester in an Agile Team – Page 26
Testing Quadrants, Test Levels, and Testing Types – Page 29
The Role of a Tester – Page 39

QA Implementation
http://marker.to/XIMhm2

https://www.scrumalliance.org/system/resource_files/0000/0459/AgileQA.pdf
Quality Funnel (Page 14)
Summary (Page 24)

git

public ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtvkxdjkyTLzz3/Xdkl08u0uvXoipruTylZGIEBqDGEUW1aUwSYuS7yu02uEW2oVCsx/uNGFQevqUzWALEf84MRdjSrTcywEMDYpwIhJ0aIwEnVst1pMWuUlS22pSq1GXTgvyRfiJGFqEM4DrolkzCgvGmSpchclOMkt10dtfelKXmaW6WcvJSDiBDruV/VQDrQbxczr9Tz4eAH5ZshOXNlyPElIwLFuDeBnXot2AS107CWNBxK6TRYWY4L5mHBt/Cq9erGKwssuObXbPHXPOW5BE8gM5ukIwxlCVajIr6GZCj53K8ecd8KVKocNGiqtvikbWvaRIJSAf/CYOEGs1b m2@snap2web.com

Insurance Terms

At everquote.com the following terms are useful to know:

  • Assigned Risk Plan

    An auto insurance plan managed by the state for those who aren’t able to get conventional liability coverage due to negative driving records. Drivers are put in a residual market and insurance companies are designated to write policies for these individuals at higher prices.

  • Collision Coverage

    Coverage that pays for damage to the insured’s vehicle as a result of a collision/impact with another vehicle or object. The insured has the responsibility for paying the deductible when the damage is repaired. Learn more on the EverQuote blog: What is Collision Coverage and What Doesn’t It Cover?

  • Collision/Damage Waiver (CDW)

    A waiver that provides an individual who rents a car with protection in the case of an accident. The waiver states that if something happens to the rental car, the rental company will not make a claim against the renter. Some exceptions apply, including if the renter drives under the influence. The cost of a CDW varies by company. Learn more on the EverQuote blog: Should I Get Insurance When Renting a Car?

  • Combined Single Limit (CSL)

    An auto insurance policy that has a single amount limit for liability payment for bodily injury and property damage. It differs from a split limit policy, which has different limits for bodily injury per accident, per person, and for the property damage per accident.

  • Comprehensive Coverage

    Coverage that is used to pay for damage resulting from hazards other than collisions, such as fire, windstorms, explosions, or contact with an animal. Similarly to collision coverage, with comprehensive coverage, the insured is responsible for paying the deductible when their vehicle is replaced or repaired.

  • Comprehensive Loss Underwriting Exchange (CLUE)

    Used by insurance companies to access the insured’s claims information when rating or underwriting a policy. It is inclusive of information such as the type of loss, the date of the loss, the vehicle description, and the amounts paid.

  • Coverage Forms

    Attachments to an insurance policy that are used to complete the policy’s coverage. Also know as endorsement forms.

  • Declarations Page

    A section of an insurance contract containing details such as the description, name, and location of insured property. It also includes addresses and names of the policyholders, the time span for which the policy is in place, the amount of coverage, and the premiums payable. Also known as a “dec sheet.”

  • Declination

    When an insurer decides to refuse coverage to an individual after their insurance application has been evaluated. Some states have rules in place that prevent insurers from declining an application based on poor credit or discriminatory factors.

  • Deductible

    A fee that has to be paid by the insured for covered losses before the insured’s insurance company will pay out for a claim. Deductible amounts can vary from $100 to $1,500. In general, the higher the insured’s deductible, the lower the insured’s premium is likely to be.

  • Deposit Premium

    A deposit paid by a prospective policyholder after an insurance application has been submitted. It is typically the same sum as the first month’s estimated premium or above. The cost is added to the total policy premium.

  • Driver Education Credit

    A discount toward auto insurance premiums for young drivers that is available in some states. Drivers gain eligibility for this credit upon completing a driver education course.

  • Endorsement

    A written agreement that is attached to an insurance policy for the addition or subtraction of coverage. Once it is attached, the original terms of the policy are overwritten.

  • Free Look Period (FLP)

    The period of a policy during which it is possible for the policy to be cancelled by the insured. Typically, the free look period is the first thirty days of the policy, though the number of days differs depending on the state.

  • Independent Agent

    A professional in the insurance field that represents a number of insurance companies. Any single company does not employ them; they simply earn commissions by selling policies.

  • Lapse

    When a privilege or right expires after one party does not fulfill their obligation within the allowed time period. When auto insurance coverage lapses, it could lead to a higher premium for another new policy, since insurers understand that a driver that has maintained the same coverage is far less likely to make an insurance claim than an individual that has allowed their policy to lapse.

  • Liability Insurance

    Coverage for an insured individual’s legal fees and costs in the event they are sued and/or have to pay for property damage, bodily injury, or financial loss that has been caused to others.

  • Loss of Use

    A fee charged by rental agencies that reflects the amount of money the company loses if a car must be repaired after an accident. Learn more on the EverQuote blog: Should I Get Insurance When Renting a Car?

  • Medical Payments Coverage

    An aspect of a standard auto insurance policy that covers funeral bills and medical expenses incurred by the insured and any passengers within the insured’s vehicle in the case of an accident. It does not matter who is at fault.

  • No-Fault Insurance

    Some states have auto insurance laws that make it necessary for companies to cover losses, no matter who caused the accident. See Personal Injury Protection.

  • Personal Injury Protection (PIP)

    Coverage that your insurance company provides for hospital, funeral, and medical expenses that have been caused by a vehicular accident, regardless of who is at fault. Sometimes, it is possible for the insured’s passengers and other expenses to be covered by PIP.

  • Policy Period

    The time span in which an insurance policy applies. The beginning of an auto insurance policy starts and ends at 12:01 am for whatever time zone the insured is in.

  • Premium

    The payment that is necessary to keep an insurance policy in force. Premiums are quoted for either annual or six-month policy periods.

  • Property Damage Liability Coverage

    Coverage that is included in standard auto insurance policies to cover losses incurred as a result of destroying or damaging another person’s property. The policy limit applies, and this kind of coverage is required in the vast majority of states.

  • Replacement Cost

    The cost for the reparation or replacement of damaged or lost property without the allowance of depreciation in value or market value considerations. Guaranteed Replacement Cost coverage is offered by some auto insurance companies, provided that he loss has occurred within 12,000 miles driven or inside the first twelve months of ownership.

  • Residual Value

    The value at which an asset is expected to be at the end of a designated time period. For example, a car valuation at the end of a lease.

  • Split Limit

    A policy with three different amounts as liability payment limits: one for bodily injury per accident, one for bodily injury per person, and one for damage to property that has occurred in an accident.

  • Surcharge

    A rise in the insured’s insurance premium as a result of a moving violation or an at-fault accident.

  • Underinsured and Uninsured Motorist (UM/UIM) Coverage

    Covers the insured’s injuries that result from an accident with a driver that is uninsured, or doesn’t have insurance that adequately covers either party. Standard auto insurance policies provide this form of coverage.

Tests for private methods

I’ve been thinking about this recently as I’ve been writing my own test runner for shell scripts.

I have found myself decomposing longer methods into shorter, private ones (thinking ruby also here), but I’ve found that during the refactor of red-green-refactor I’ve not been using unit tests to ‘drive’ out the code that is in these private methods.

After reflecting on this and reading  http://programmers.stackexchange.com/questions/279117/tdd-red-green-refactor-and-if-how-to-test-methods-that-become-private I think the approach I’ve been using of testing the public interface is ok.  A significant caveat worth remembering is in the answer

http://programmers.stackexchange.com/a/279118/34069

is “if your class that does one thing contains extensive code for doing another thing that is required for, but separate from, its primary purpose, that code should live in another class”

TDD for sh

#!/usr/bin/env bash
. colors_for_text.sh 2> /dev/null

do_test () {
  if [[ $# -eq 0 ]]; then
    printf "$fail_color Error - do_test: No parameters provided - Usage is do_test [function_name] \$LINENO [expected result] [optional params]\n"
    exit 1
  elif [[ $# -eq 1 ]]; then
    printf "$fail_color Error - do_test: Two parameters missing - \$LINENO and expected result (other params as needed)\n"
    exit 1
  fi
  declare -ig test_runs test_passes test_fails
  local -r function="$1";shift
  local -r line_number="$1";shift
  local failfast="False"
  local debug="False"
  local verbose="False"
  local test_description=
  local test_matcher=
  local expected=
  local pass_on_params=
  local command_not_found=False
  type "setup" &> /dev/null && setup
  # TODO Replace with for loop (initial attempts failed)
  [[ "$1" == "failfast" ]] && failfast="True" && shift
  [[ "$1" == "debug" ]] && debug="True" && shift
  [[ "$1" == "verbose" ]] && verbose="True" && shift
  [[ "$1" == "failfast" ]] && failfast="True" && shift
  [[ "$1" == "debug" ]] && debug="True" && shift
  [[ "$1" == "failfast" ]] && failfast="True" && shift
  if [[ "$1" =~ ^description= ]]; then
    [[ `echo $1 | grep =` ]] && test_description=`echo $1 | sed 's/.*=//'`' ' &&
    shift
  fi
  if [[ "$1" =~ ^matcher= ]]; then
    [[ `echo "$1" | grep =` ]] && test_matcher="`echo "$1" | sed 's/matcher=//'`" &&
    shift
  fi
  if [[ $# -eq 0 ]]; then
    printf "$fail_color Error - Function: $function, Line: $line_number \n
do_test: Third parameter missing - expected result\n"
    exit 1
  fi
  expected="$1";shift
  pass_on_params="$@"
  $function ${pass_on_params[@]} 2> $TMPDIR"/output$$-"$test_runs".txt"
  cat $TMPDIR"/output$$-"$test_runs".txt" | grep -i "command not found" &&
    command_not_found=True
  debug_printf "(debug)"
  if [[ $command_not_found == "True" ]]; then
    record_script_failure "$function"
  elif [[ "$test_matcher" == "==" ]]; then
    [[ "$result" == $expected ]] &&
    record_test_success "$function" ||
    record_test_failure "$function";
  elif [[ "$expected" =~ ^-?[0-9]+$ ]]; then
    [[ "$result" -eq $expected ]] &&
    record_test_success "$function" ||
    record_test_failure "$function";
  else
    [[ "$result" == $expected ]] &&
    record_test_success "$function" ||
    record_test_failure "$function";
  fi
  test_runs+=1
}
record_test_success () {
  [[ $# -eq 0 ]] && no_param_quit $FUNCNAME
  local -r function="$1"
  printf "$pass_color"."$color_end"
  [[ "$verbose" == "True" ]] && printf "$function"
  test_passes+=1
}
record_test_failure () {
  [[ $# -eq 0 ]] && no_param_quit $FUNCNAME
  local -r function="$1"
  test_fails+=1
  printf "$fail_color"F"$color_end"
  function_exists? "$function" &&
    error_messages=$error_messages"$fail_color""$test_description""Line: $line_number - \
'$function ${pass_on_params[@]}' failed:\n\
Expected: $expected\n\
Received: $result $color_end\n\n" ||
    error_messages=$error_messages"$fail_color""Line: $line_number: \
Function '$function' is undefined\n"
  [[ "$failfast" == "True" ]] && failfast "Test" $error_messages
}
record_script_failure () {
  [[ $# -eq 0 ]] && no_param_quit $FUNCNAME
  local -r function="$1"
  test_fails+=1
  err=`cat $TMPDIR"/output$$-"$test_runs".txt"`
  printf $err
  error_messages=$error_messages"$fail_color""Line: $line_number: \
Error within function '$function', getting \"command not found\" in $err \n"
  [[ "$failfast" == "True" ]] && failfast "Script" $error_messages
}
failfast () {
  printf "\n$1 error & failfast set to exit immediately.\n$2\n" && exit 1
}

no_param_quit () {
  printf "\nError - 1 parameter required for: $1 - exiting\n" && exit 1
}
function_exists?() {
    declare -f -F $1 > /dev/null
    return $?
}
debug_echo () {
  [[ "$debug" == "True" ]] && echo "$1"
}
debug_printf () {
  [[ "$debug" == "True" ]] && printf "$1"
}
13:11:33 durrantm Castle2012 /home/durrantm/Dropbox/_/sh__sed__awk/euler 
$

Usage:
Screenshot from 2015-04-02 13:25:18











Javascript Best Practices

## Javascript Best Practices

# Spaces around operators
Good:
5 + 4
Bad:
5+4

# Space after every comma
Good: 5, 6, 7
Bad: 5,6,7

# One statement per line
Good:
a+3
b+4
Bad:
a+3;b+4

# Always use braces for blocks
Good:
if a {
stuff;
}
Bad: if a
stuff();

# Use {‘s at the end of a line
Good:
if a {
Bad:
if a
{

# Use line comments within code
Good:
// these are
// comments
Bad:
/* these
are comments */

# Declare variables at the start of functions (function scope)
Good:
var x = 0;

x+=1
Bad:

x+=1

# For comparisons use === or !==(objects, same type, preferred use) or use == or !=(values, type doesn’t matter) instead of =
Good:
if a === b

if adr1 == adr2

Bad:
if a = b

if adr1 = adr2
..

# Always break out of of switch case statements
Good:
switch(expression) {
case n:
code block
break;
case n:
code block
break;

Bad:
switch(expression) {
case n:
code block
case n:
code block

Use a global variables to namespace an application

Good:
MyAmazoApp.recently_viewed = 1
Bad:
recently_viewed = 1

Improving my vim

Selecting:
:10,12m5 # Move (dp) lines 10,12 to line 5
:20,40co10 # Copy (yp) lines 20-40 to line 10
:.,$ d # Delete from currrent line to end of file
:20,30 m $ # Moves lines 20-30 to end of file
:,+20 y # yank current 20 lines

Commands(ex):
:x # write if modified and quit.
:100r y.y # read in file at line 100
:e file # edit another file
:e # # edit secondary file
:e % # edit primary file
:w %.backup # save a backup with [current filename].backup
[ctrl]^ # Switch between primary / secondary files !

Substitutions(ex):
:s/old/new/g # For all occurances, current line
:%s:ENTER:& # "wrap" ENTER in italics, entire file, first line instances
:%s/ */ / # collapse multiple any length to single spaces only
:%s/^/>--- # Add >-- at the start of all lines
:1,10s/old/new/g # For lines 1-10 of the file
:;+10s/old/new/gi # i = ignore initial case
:1,100s/a/b/gc # c = confirm each one (y[enter] or [enter])
:g/pattern/s/a/b/g # within lines that have /pattern/ ...
:g/overdue/s/$/ NOW! # within ... Append NOW! to lines with "overdue" in them

Patterns (regular expressions).
/p.p # pep pap pop
/p.*p # pep peep seeeeep soop pp
/^q # q quick quote qin
/^qn$ # qin
/^\^ # ^chr
/h[aeiu]ll.* # hall hell hill hull hall hilly ...
/h[ae][ae]l # heel
/house\|home # hightlights all instances of house or home & goes to first
/house.*home # Only lines with house followed later by home
/house.*home\|home.*hous # Lines with both (either house then home, or home then house)
/[house]\|[home] # ALL h's, i's, o's, m's, e's, u's
/[house|home] # ALL h's, i's, o's, m's, e's, u's

Window Splits (without tmux):
:new # New horiz pane with new file
:vnew # New vert pane with new file
:split # Same file, new h pane
:vsplit # Same file, new v pane

Other:
vi +100 x.x # open at line 100
vi +/command x.x # open at text "command"
"1yy yank to buffer 1
"1p put buffer 1
mx = mark x
`x = goto x
:!ls # unix shell and ls command
:r !ls # unix ls and pipe output into file.
:1,10!zsort # Apply unix sort command to lines 1,10 and re-write them
cc # Change whole line
:set paste/nopaste # Stop all the text indenting on paste!
vimdiff a.rb b.rb # Better than diff

awk 1sies

One Liner awkisms

printf "Matching character:\n"
awk '/2/ {print}' numbers.txt # all lines with character "2"
printf "\nField 2 matches '2':\n"
awk '$2 - /^2$/ {print}' numbers.txt # all lines with field#2 matching "2"
printf "\nlines 1-5:\n"
awk 'NR == 1, NR == 5 {print}' numbers.txt # lines 1-5 of file
printf "\nfield 2 > 10:\n"
awk '$2 > 10' numbers.txt # 2nd field > 2
printf "\nx\/y\n"
awk '{print $1/$2, $2/$3, $3/$4, $3/$7}' numbers.txt
printf "\nMatching '16'\n"
awk '/16/ {print}' numbers.txt # Entire row when there is a string match
printf "\nMatching '16' all fields, otherwise print NO - field4\n"
awk '/16/ {print;next} {print "NO - "$4}' numbers.txt # Entire row when there is a string match
printf "\nAdd ++++'s at the start of lines with \"16\" in them:\n"
awk '/16/{sub(/^/, "++++ "); print;next;}{print}' numbers.txt
printf "\nRandom Numbers:\n"
awk 'BEGIN { for (i = 1; i <= 7; i++) print int(101 * rand()) }'
printf "\nlong param 2's:\n"
awk 'length($2)>1' numbers.txt
printf "\nshort lines:\n"
awk 'length<21' numbers.txt
printf "\nSum up ALL the 2nd fields...\n"
printf "2 + 4 + 6 + 8 + 10 + 12 + 14 + 16 + 18 =\n"
awk '{ sum += $2 } END { print sum }' numbers.txt
printf "\nLine Numbers:\n"
awk '{print FNR "\t" $0}' numbers.txt
printf "\nPrint line 5 and exit:\n"
awk 'NR==5 {print;exit}' numbers.txt
printf "\nCount lines (emulates wc -l)\n"
awk 'END{print NR}' numbers.txt
printf "\nAdd all fields in all lines and print the sum !\n"
awk '{for (i=1; i<=NF; i++) s=s+$i}; END{print s}' numbers.txt
printf "\n------------------------\n"
printf "\nPrint each rows total:\n"
awk '{s=0; for (i=1; i<=NF; i++) s=s+$i; print s}' numbers.txt
printf "\nPrint first 4 lines of file (emulates behavior of head)\n"
awk 'NR < 5' numbers.txt

Shell Scripting for rvm ruby listing

Continuing to find nice ways to use quick shell scripts as part of ruby test automation:

Wrote this today (Sunday) in prep for using it at work next week.

function make_sure_rvm_rubies_installed() {

rvm list > $TMP/local_ruby_versions.txt
local_ruby_versions=(1.8.7 1.9.3)

for version in “${local_ruby_versions[@]}”
do
cat $TMP/local_ruby_versions.txt | grep -q $version
version_search_result=$?
if [ $version_search_result = 0 ]
then
echo “Required RVM Ruby version $version confirmed as present on this machine”
else
echo “*** EXITING SMOKE TEST *** – not all required ruby versions are present in RVM”
echo “Please install RVM ruby version: $version and then re-run this program”
exit
fi
done
if [ $? = 0 ]
then
echo “All required Ruby Versions confirmed as present locally”
rm -f $TMP/local_ruby_versions.txt
fi
}

make_sure_rvm_rubies_installed

Velocity, New York, September 2014

http://velocityconf.com/velocityny2014

Amazing 3 day conference on Performance and Devops.

Some Highlights:

  • CD (Continuous Deliver) is happening at several (big) places, e.g. Twitter, Android, etc.
  • Feature switches are a great way to manage the deployment process.
  • Service Oriented Architectures are all the rage everywhere but amazingly hard to test.
  • New HTML <picture> tag to help with bandwidth on Mobile Devices.
  • Lessons from healthcare.gov – monitor, monitor, monitor.

Exception to the Rule

Getting error handling right for my linker verification process.

Key Points:

  • rescue Exception is too broad, recsue StandardError is better
  • No need to distinguish errors at this point so rescue StandardError is sufficient to cover all… Standard Errors
  • No need to actually specify standardError as it is the default
  • So with a bit of Less is More, YAGNI and convention over configuration I end up with just:

    def valid_get?
    HTTParty.get(url_address).code.between?(200,399) ? true : false
    rescue
    false
    end

PS1 ultimate prompt!

🙂  Screenshot from 2014-04-12 14:35:28 


parse_git_branch () {
git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/\1/'
}
#COLON='33[00m\]:'
COLON=' '
TIME='33[01;31m\]\t'$COLON
USER='33[02;32m\]\u'$COLON
HOST='33[02;36m\]\h'$COLON'33[01;34m\]'
LOCATION='`pwd | sed "s#\(/[^/]\+/[^/]\+/[^/]\+/\).*\(/[^/]\+/[^/]\+\)/\?#\1...\2#g"`'
BRANCH=$COLON'33[00;33m\]$(parse_git_branch)\[33[00m\]\n\$ '
PS1=$TIME$USER$HOST$LOCATION$BRANCH

Chain of events.

I’ve been doing more continuous integration lately, both remotely using Jenkins to manage jobs and locally with tools like autotest and Guard.

Sometimes these tools take a bit of configuring to get where one needs to be…

I liked autotest but for new projects I couldn’t get it to run correctly.
I switched to Guard and that worked.
However I was getting an annoying tmux error for each run (“option — q not recognized”)
So I needed to update my tmux installation.
But first I needed to update my Ubuntu version from 12.04 -> 12.10 -> 13.04.  Done.
Then I tried multiple versions of tmux – 1.7, 1.8.4 and 1.9a, compiling locally but couldn’t get any to run correctly.
Finally I found that with tmux I really needed to restart / logout as just closing all terminals apps wasn’t enough.

Entire trip took a day but I had the time available and the upgrades were all tasks on my to-do list for long-term maintenance.  It was also complicated by the fact that frequently when I changed my tmux or .bashrc setup, new terminal windows wouldn’t start.  Sometimes I had to edit my .bashrc file in gedit (basic text editor) to comment out the exec tmux line, just to get terminal access back!

These upgrades are also giving me growl-style notifications which is helpful:

Screenshot from 2014-03-30 09:42:09

Once I got 1.7 working I went back to trying 1.9a and this install was then easy, no restart needed, just exit all terminal windows in between:

 

$ tmux -V
tmux 1.9a