AWS Developer

Advertisements

AWS Architect

How to test and what to test for an API

At a high level

Test the API Endpoints, Status Codes and Data

At a detailed level one needs to ask the following questions.

The answers will guide what and how to test.

• What documentation exists ?
• What functionality it provide ?
• Does it support concurrency ?
• What are the API endpoints ?
• Is the API internal or external ?
• Which endpoints are idempotent ?
• Are endpoints stateless or stateful ?
• Do any workflows*1 vary by client ?
• Are there performance requirements ?
• Do API endpoints make up a workflow ?
• What validations are expected for data ?
• What system or library is behind the API ?
• Do we need to mock dependent services ?
• Does it constrain traffic aka Rate Limiting ?
• What (if any) versioning approach is used ?
• Does the API support Multiple Languages ?
• If already using SOAPui, how is it integrated ?
• Is the API be restricted to a country or region ?
• Does it provide client stubs in specific languages ?
• What status codes are expected for given endpoints ?
• What domain format and structure exists for the data ?
• Does the API use HATEOS*2 for self documentation ?
• What kind of data validation/ testing can be performed ?
• What API is supported by the test framework I’m using ?
• What actions are performed, e.g. GET, PUT, POST etc ?
• Do we need to prepare dependent test data or services ?
• What non-API approaches will be needed to verify data ?
• Are there existing API definitions e.g. WADL, WSDL, Thrift ?
• What non-API approaches will be needed to prepare data ?
• What (if any) Authorization (‘what’) mechanism will be used ?
• What (if any) Authentication (‘who’) mechanism will be used ?
• Who will use it, external programmers or another internal module ?
• What format(s): SOAP, REST, GraphQL, Thrift, ProtoBuffer, Other ?

*1 Workflows often require multiple API calls and may have dependencies between them
*2 HATEOS – Hypertext As The Engine Of Application State, which allows self-discovery of an API

Credit to https://sqa.stackexchange.com/a/23693/8992 whose focus was performance testing.

Beautiful Questions

15 faves

• What Else ?
• How can I help ?
• Can I begin now ?
• Do you know why ?
• What would you do ?
• What could I change ?
• What is your opinion ?
• What can I stop doing ?
• What do you like least ?
• What am I so afraid of ?
• Do I reflect and ask why ?
• Do I ask Why before How ?
• Can I take connect breaks ?
• What would an outsider do ?
• Do I admit being wrong frequently ?

HDI – Halifax Developer Initiative

Intensive Study Program for key knowledge in current technologies

Time for more js with Node and Typescript

My Geek Life

2019 Recommendations

Declarative vs Imperative

A nice simple example

Declarative

Choose all the items you want

Imperative

Check the checkbox in the first column of each row for the rows you want

Page Objects – Duplicates and Orphans

Having Page Objects is great but a couple of issues often show up:

• Duplicates – where there is more than one definition
• Orphans – where no spec refers to them anymore

To prevent this I wrote the following rspec tests.

They are based on having locators in a locators.yml file, e.g.

`​​locator: ‘some_identifier’`

and on test files being in the format *_spec.rb

It support tests in subdirectories

require 'rspec'

describe 'Page Objects locator yml file' do

  before :each do
    load_locators
  end

  it 'does not have duplicates' do
    if @keys.uniq.count != @keys.count
      dupe_keys = @keys.select{|n| @keys.count(n) > 1}.uniq
      dupe_keys.each do |key|
        @pairs.each do |pair|
          p "#{key.to_sym} : #{pair[key]}" if pair[key]
        end 
      end
    end
    
    expect(@keys.uniq.count).to (eq @keys.count),
      lambda {"Duplicate page object keys found! #{dupe_keys}"}
      
  end 

  it "uses all its locator keys" do
    files = Dir.glob("**/*_spec.rb")
    unused_keys = []
    @keys.each do |key|
      @key_used = false
      files.each {|file| search_file_for_key(file, key) }
      unused_keys << key unless @key_used
    end
    unused_keys_exist = unused_keys.size > 0

    expect(unused_keys_exist).not_to be,
      lambda {"Failure - orphan page objects  exist #{unused_keys}"}
      
  end 

  def search_file_for_key(file, key)
    spec_file = File.open(file) 
    file_contents = spec_file.read
    spec_file.close 
    @key_used = true if file_contents.match(/#{key}/)
  end
  
  def load_locators
    locators_file = File.open('locators.yml')
    @pairs = []
    @keys = []
    
    locators_file.each_line do |line|
      words = line.split(': ') 
      @pairs << {words[0] => words[1]}
      @keys << words[0]
    end
    locators_file.close
  end
  
end

What does debugging ruby look like?

It’s hard to capture the look of “work in progress” when it is code.

It’s usually a fleeting moment.  One is ‘in the moment’ and when one print out and checks enough stuff to understand what’s going on, all the ‘aiding’ code is removed.

Here’s the somewhat  intimate view of actual wip with lots of prints and assertions of various kinds to see if what I expect is going actually is going on, because initially it certainly wasn’t

There’s some ugly stuff here, soon removed after capturing it here, but here it is before I make it look all pretty again.

Writing good code, step by step

It’s hard to capture the process of writing code. The way you mold and improve it.

I think of it as often being the ‘anything works and ugly code is ok then immediately refactor’ approach.

Here are 4 versions of ‘Rock, Paper, Scissors’ showing such code molding

p 'v1-------'
# Version 1
my_guess = rand(3)
your_guess = rand(3)
my_guess_text = ['rock', 'paper', 'scissors'][my_guess]
your_guess_text = ['rock', 'paper', 'scissors'][your_guess]
p "You guessed #{your_guess_text} and I guessed #{my_guess_text}"
winner=''
case my_guess_text
when 'rock'
if your_guess_text == 'paper'
winner = 'you'
elsif your_guess_text == 'scissors'
winner = 'me'
end
when 'paper'
if your_guess_text == 'scissors'
winner = 'you'
elsif your_guess_text == 'rock'
winner = 'me'
end
when 'scissors'
if your_guess_text == 'rock'
winner = 'you'
elsif your_guess_text == 'paper'
winner = 'me'
end
end
winner == '' && winner = 'tie'
p "Winner was #{winner}"

p 'v2-------'
# Version 2
my_guess = rand(3)
your_guess = rand(3)
my_guess_text = ['rock', 'paper', 'scissors'][my_guess]
your_guess_text = ['rock', 'paper', 'scissors'][your_guess]
p "You guessed #{your_guess_text} and I guessed #{my_guess_text}"
who_wins = {paper: 'rock', rock: 'scissors', scissors: 'paper'}
result = who_wins[my_guess_text.to_sym]
winner =
if my_guess_text == your_guess_text
"tie"
elsif your_guess_text == result
"me"
else
"you"
end
p "Winner was #{winner}"

p 'v3-------'
# Version 3
my_guess = rand(3)
your_guess = rand(3)
my_guess_text = ['rock', 'paper', 'scissors'][my_guess]
your_guess_text = ['rock', 'paper', 'scissors'][your_guess]
p "You guessed #{your_guess_text} and I guessed #{my_guess_text}"
winners = {paper: 'rock', rock: 'scissors', scissors: 'paper'}
winner =
if your_guess_text == my_guess_text
"tie"
elsif your_guess_text == winners[my_guess_text.to_sym]
"me"
else
"you"
end
p "Winner was #{winner}"

p "v4------"
# Version 4
winners = {paper: 'rock', rock: 'scissors', scissors: 'paper'}
my_guess = winners.keys[rand(3)]
your_guess = winners.keys[rand(3)]
p "You guessed #{your_guess} and I guessed #{my_guess}"
winner =
if your_guess == my_guess
"tie"
elsif your_guess == winners[my_guess]
"me"
else
"you"
end
p "Winner was #{winner}"

Puzzle fix #2

Trying to get Docker to use nokogiri

Posted the steps I took at https://stackoverflow.com/q/55678669/631619

in the end I figured out I needed to add

RUN apt install -y build-essential patch ruby-dev zlib1g-dev liblzma-dev

to the Docker build

Puzzle fix #1

My attempt to capture the minutia of a programmers life

Working with ruby, switched laptops and out of the blue

$ bundle
Traceback (most recent call last):
2: from /home/michael/.rbenv/versions/2.5.1/bin/bundle:23:in `<main>'
1: from /home/michael/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rubygems.rb:308:in `activate_bin_path'
/home/michael/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rubygems.rb:289:in `find_spec_for_exe': can't find gem bundler (>= 0.a) with executabl
e bundle (Gem::GemNotFoundException)

hmmm. and here we go:

What version of ruby do I have (and make sure if I have _some_ version!)

$ ruby -v
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]

hmmm, can I install gems?

$ gem install pry
Successfully installed pry-0.12.2
Parsing documentation for pry-0.12.2
Installing ri documentation for pry-0.12.2
Done installing documentation for pry after 1 seconds
1 gem installed

hmmm, yes. So let me try bundle again in case one time issue, network, etc.

$ bundle
Traceback (most recent call last):
2: from /home/michael/.rbenv/versions/2.5.1/bin/bundle:23:in `<main>'
1: from /home/michael/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rubygems.rb:308:in `activate_bin_path'
/home/michael/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rubygems.rb:289:in `find_spec_for_exe': can't find gem bundler (>= 0.a) with executabl
e bundle (Gem::GemNotFoundException)
08:33:20 michael durrantm-2018 /home/michael/Dropbox/_/ultimate-weather-org/ultimate-weather-ruby master

hmmm, maybe reinstall bundler ?

$ gem install bundler
Fetching: bundler-2.0.1.gem (100%)
Successfully installed bundler-2.0.1
Parsing documentation for bundler-2.0.1
Installing ri documentation for bundler-2.0.1
Done installing documentation for bundler after 2 seconds
1 gem installed

ok, let me try again…

$ bundle
Fetching gem metadata from http://rubygems.org/..........
Fetching public_suffix 3.0.3
Installing public_suffix 3.0.3
Fetching addressable 2.6.0
...

ok, that worked. fairly simple one this time…

 

 

Tech Refresh Roundup, Spring 2019

Over the next month I will post daily activities here to reflect my intent to refresh myself on various technologies.

 

3/26 Wednesday C++ and pack of cards routines.
3/27 API development with Postman – learn more Postman basics
3/28 API development with accuweather – got account and key

 

Tests as documentation

I have found that rspec’s formatting just lacks options.

The output is critical given that I put it in front of business users.

Here’s my way to do that.  I will also add other domain specific lines for the codebase I am actually working on but the ones below are intended to apply to all uses of rspec:

find . -regex \
 '\(.*_spec\.rb\|spec_.*\.rb\)' ! -name 'spec_helper.rb' \
 -exec cat {} \; | # cat out each file \
grep -E \
'(^ +?describe |RSpec\.describe|^ +?it |^ +?expect|\.should)'|\
sed -E 's/ do +?$//' | # No need for the do \
sed 's/, order\: \:defined//' | # No need for order: :defined \
sed "s/\.value)/)/" | # No need for .value \
sed "s/ eq / equal /" | # Change 'eq' to 'equal' \
sed "s/ eq(/ equal (/" |
sed "s/).to_s/)/" | # No need for .to_s\
sed "s/.text)/)/" | # No need for .text \
sed "s/RSpec\.//" | # Remove Rspec. \
sed $'s/^describe /\\\ndescribe /' | # linefeed between specs \
sed "s/(page\./(/" | # No need for page. \
sed -E "s/find_by_id ?//" | # No need for find_by_id \
sed -E "s/\(\(/(/;s/\)\)/)/" | # Remove double parens ((and))  \
sed -E "s/\(find /(/" | # No need for find statements \
sed -E "s/\(p\./(/" | # No need for p. before page objects \
sed -E "s/have_css/have/" | # make have_css more readable \
sed -E "s/\.disabled\?\)\.to/.disabled\? to/" | # Fix parens \
sed -E $'s/enddescribe/end\\\n\\\ndescribe/' | # Fix issue \
sed '/^end$/d' | # Remove lines with just end on them \
sed '1d' # Remove first (blank) line

Building an Iron pipeline – in the cloud

Indy Agilists build a devops pipeline… in a day

Indianapolis, Indiana, Saturday 01/05/2018

As the small crowd started showing up on a crisp Saturday morning In January for the ‘hackathon at the Ironworks’ there was a lot of uncertainty about just what they would – or even could – actually “build”.
Everyone knew each other because they worked at or was a contractor at Sallie Mae and had seen some of the pieces of this at work. This was an opportunity for them to build a similar code testing and deployment system using modern secure open source tools and using some of the most modern hosting and application building services that are pre-made to be plug and play with each other. Using personal (non-sallie mae) pc’s and a non-sallie mae network were obvious prerequisites to the endeavor.

So how’d they do?

Well clearly fun was on the Agenda from the start. The Admin Assistant gave out high quality (metal) spinners to all and a stuffed polar bear took center piece on the table (see photo). Then folks divvied up the various roles. Bascially nearly everyone was a Director! There was a Director of Application Development, Director of Devops, Director of Quality Integrations, Director of Product Development and of course… an actual engineer… with the title ‘Actual Engineer’. The person behind the hackathon itself took on the highest title of course – Admin Assistant.

Did they actually build the Pipeline?

Yes! Within a few minutes they had a slack workspace, connected to a Jira ticketing system. From that they were soon able to hook in code reviews and secure Continuous Integration with CircleCI to run tests in the cloud. As the day progressed they were then able to scale the system up to use 4 parallel servers to run the tests and a Code Grading system to make code quality an easy and visible process. By the time they were finished they were able to do the end to end process of make a change, perform code review and grading, run all the tests in the cloud and if passed, promote the code to production – a true CI/CD system.

What did they learn?

Many of the integrations between the different vendors and services were remarkably easier to configure, often in literally seconds or minutes. It was a stark comparison to many enterprise systems that have traditionally required months of pain staking work to set this up. The exercise was also a good example and reminder of the basic of running tests in the cloud as the default way of working.  Perhaps most of all the power of working together in the same location with the ability to simply ask each ‘director’ to do the work (often relatively small with modern tools) right on the spot was wonderful to experience.

Jira Juggling

Re-entering the world of Jira I once again encountered mapping and transition issues.

The import lessons were to remember to consider System Level workflows, Project Level workflows and Board level mappings.  The three work together in a non-obvious way.  Add to that the need to understand active and inactive workflows, workflows vs. workflow schemas and the Jira approach to publishing changes, and you quickly get lost in a series of dead-end error message such as ‘can’t edit an active workflow’ and the inability to be able to move cards to different status on the board (a common result of not getting changes right).

Also one key thing is to make sure to refresh screens and boards wile doing changes.  Some changes are updated real time, others require a page refresh.  Easy to get caught out thinking a change hasn’t worked when it has.

Crucial screenshots:

1. These are the system level controls for things like workflows.  They are often not obvious or easy to know about when you are at a project or board level

2.Workflows and Workflow Schemas and which are active in projects

3. When you change workflow you’ll frequently need to then readjust a board to deal with the different status and how they should show.

4. At the project level there are settings

5. Including workflows

6. Different boards may need different status mapping to get their workflow right

 

Terms of Endearment

Actually the term I am thinking off is far from ‘endearment’ but let me avoid naming it for a minute ‘cos it is a pretty loaded term at this point.

Here’s the behaviors I observe in my current work:

• When help is needed from another team I ask my manager and they ask the other persons manager and the other persons manager then directs the other employee to do the work
• When I disagree with someone on another team I let them know what my position is and who I work for and tell them that they will have to contact my manager if they disagree with my decisions
• When I can’t make a deadline I will identify the team or person that is blocking me so that my boss knows about it and can take action
• Periodically management will need to send a ‘clear message’ to the team about what their priorities should be.  Often done in an urgent fashion generating fear

The one thing you don’t see here is any mention that management is using command and control.  Also there is no mention of whether the manager is deeply involved in the conversations or just watching from the sidelines.

What can be said however is that the command and control is clearly taking place and to a large part due to the employee enabling their manager.  By asking the boss to ‘fix things’ instead of asking to (and being able to) work directly with the other employee a huge amount of non-essential work emerges, often including backlogs that are weeks long (even if the current request is simple to do), and all the trapping of command and control and the politics that naturally come with it.

This is why part of a new culture needs to be people asking other people directly for help without groups hiding behind the shields of SAFe, Ticket systems like ServiceNow, CommSec or whatever process is used to create lags.  The lags can easily end up taking 99% of the elapsed time to address the need.  This defeats the ability to go faster and without that the business is in jeopardy.

GoodAt BadAt

An novel and open look at what I am good at and bad at.

Super simplistic for sure but I am exploring this as an interview tool that allows a candidate (such as myself) to self-assess and reduces the need for companies to need to test them for skills they openly admit being bad at.  This is a new approach.  Might not work for some.  Focusing on pushing candidates on their personally rated GoodAt list may be a helpful approach.
For me, the ‘good at’ list also reflects the skills I’ve actually needed in my automation and leadership positions in several recent jobs and the ‘bad at’ are skills I’ve not needed to use in those positions.

Good At

Good At

Bad At

The Journey

My journey for knowledge and transformation started 18 months ago as I joined a new and large company.  As I started to read books about new ways of working I found that being at an organization going through dramatic change was the best place to be in order to fully explore many of these issues in person.

So I started reading.
and reading.
and reading.
and reading some more.

After reading a few influential books and getting an ever greater hunger for more, I started to notice something: my thinking had changed.  My approach to problems had changed.  omg my marriage had changed.  My health changed dramatically.  At first I didn’t see the changes but then a good friend pointed out “have you noticed how there seems to be a cumulative effect just from reading so many of these books?”.  Boom.  She was spot on (thanks Stacey!).  Once I started to become aware of this – and upped my reading volume even more I started to become more and more aware of this.

It is one things to read a book.  Or two.  or three.  However good they are, they tend to have a fairly limited effect on you.  You read some great things and then ‘life continues’.  Some of them you will retain and they will pop up at the perfect time in the future, but for the most part, much of the advice fades into the background of life.

Unless.
You read a lot.  You start spending a significant portion of your week just reading, highlighting and thinking.  Then more dramatic transformation can happen.  We have learned that the brain is plastic and this feels to me like taking advantage of that to change and ‘reprogram’ yourself.

Having gone through this I want to pass on what I’ve learned. I want to show the path I’ve taken and how it can work for you in far fewer steps.

OK so I have a list of 80 books on transformation and leadership.  All you’ve got to do is go through all those book, highlight and reflect the key parts, and… JUST KIDDING.
I have actually read 80 books about change on my journey but if I had to do the journey again I would pick out the super critical ones.  So this is what I have done for you.  At first I picked the top 40.  So yeah, right, just read 40 books.  Again, who am I kidding?  So OK I chopped that down to 20.  Still too many huh?  OK I chopped that down to 15.  Still too many?  Tough.  That is really the minimum I can do (I tried 12 but it meant critical books on the cutting room floor).  I think every one of these books is a critical part of the journey and not to be skipped.

The other interesting thing for me was the order I read those book in (preserved here).  One book lead to another – primarily through Amazon recommendations from the previous book(s) – but also from recommendations from current colleagues – thanks Joe, Carla and Stacey – and from the challenges I faces in my daily work at a company going through and struggling with an Agile transformation.

So here is the list in the order that I read them:

1. The Psychology of Computer Programming
   Gerald Weinberg
   The book that ‘got me started’.  I was looking for some unusual reading and this caught my eye.  It seemed such a different take from all the programming books I’d read over the decades but I though I’d give it a go.  It also seemed somewhat dated perhaps.  Once I started reading it however I quickly realized that the advice was priceless and just as relevant today as when it was written.  This was the start of my journey…
   

   ---

2. Crucial Conversations Tools for Talking When Stakes Are High
   Kerry Patterson
   This made me consciously aware of so many things about the conversations I have had at every place I have worked at, but had never paid attention to before in detail.  It was eye opening to me
   

   ---

3. Managing the Unmanageable
   Mickey Mantle
   This opened my eyes to the characteristics I had observed my whole working career about how ‘techies’ work and how it can be so different from other professions.
   

   ---

   The next three book are all about experiences from specific companies and helped me to start having some answers to the many questions that I was coming up with in my current workplace.

   ---

4. Radical Candor: Be a Kick-Ass Boss Without Losing Your Humanity
   Kim Scott
   The story told here resonated strongly with me.  Being open and honest is explored in a wonderful way in this book
   

   ---

5. Powerful: Building a Culture of Freedom and Responsibility
   Patty McCord
   This is story of Netflix and new ways of working that I strongly related to
   

   ---

6. Work Rules!: Insights from Inside Google That Will Transform How You Live and Lead
   Laszlo Bock
   Who wouldn’t want to know how they do it at Google huh?  The interesting thing I found in this book was the emphasis over and over again, that it doesn’t matter what the business or industry is, the things that google does to reward and retain talent can be done anywhere
   

   ---

7. Accelerate: The science of Lean Software and DevOps
   Forsgren PhD
   A great read on what going faster means in practice
   

   ---

8. The Culture Code: The Secrets of Highly Successful Groups
   Daniel Coyle
   A great book that examines what a change in culture really means
   

   ---

   At this point in my journey I started to get a lot more ‘into’ how people and groups work (and don’t) work well to achieve goals and also some of the science behind it.
   I started to read a lot of books on just how people work together and what has proven to work well and what hasn’t.

   ---

9. Alive at Work: The Neuroscience of Helping Your People Love What They Do – Daniel M. Cable
   This was a fascinating book on the science behind how the brain works
   

   ---

10. Humble Leadership: The Power of Relationships, Openness, and Trust
    Edgar H. Schein
    An eye-opening book that goes by beyond ‘servant’ leadership to a radically different work of working.  I want to work with people who have read and agree with this book.
    

    ---

11. Not Nice: Stop People Pleasing, Staying Silent, & Feeling Guilty… And Start Speaking Up, Saying No, Asking Boldly, And Unapologetically Being Yourself – Aziz Gazipura
    This book’s title may mislead you
    It is about being nice by being honest and it is a great way to think differently.
    This is the book that affected (improved) my health the most by explaining the stress that I hadn’t been able to see or understand well.
    

    ---

12. Psychological Triggers: Human Nature, Irrationality, and Why We Do What We Do. The Hidden Influences Behind Our Actions, Thoughts, and Behaviors
    Peter Hollins
    This gives you a much greater understanding of the why behind peoples motivations and actions
    

    ---

13. Dare to Lead: Brave Work. Tough Conversations. Whole Hearts
    Brené Brown
    A moving book that brought me to tears.  Amazing advice for tough talks
    

    ---

14. Perfect Software and other illusions about testing
    Gerald Weinberg
    Another fantastic book by Gerald that is a most read for anyone doing testing and automation or leading that effort.
    

    ---

15. Bring Your Human to Work: 10 Surefire Ways to Design a Workplace That Is Good for People, Great for Business, and Just Might Change the World
    Erica Keswin
    Continuing the ‘make it real’ movement at work with tips and tools to succeed
    

    ---

    Finally here are all 15 books should you want to just go on a spending spree.
    I got them all on kindle so total cost was probably around $200.  A great investment.
    

 

Useful Dockerfile Setups

https://github.com/durrantm/docker_11_2018_bashful_bash

e.g. alpine

FROM alpine:latest
LABEL maintainer="Michael Durrant<junk@snap2web.com>"
RUN apk add bash git vim
COPY alpine_bashrc /root/.bashrc
COPY .bash_functions.sh /root
COPY .bash_aliases /root
COPY .git-completion.bash /root
RUN "/bin/bash"
RUN git config --global user.name 'Michael'
RUN git config --global user.email 'no-reply@google.com'

Some common docker commands:

docker version
docker images
docker pull alpine
docker ps -a
docker start container_name
docker attach container_name
docker ps

Without a Dockerfile…
docker run -it alpine
docker attach container_name

With a Dockerfile…
docker build -t alpine_plus_some .
docker run -it alpine_plus_some
docker attached container_name

Top five technologies to master by 7/1/2019

“Choose the five tools and technologies that you want to be highly proficient in by July 1st 2019”
I picked:

– Chef
– Docker
– AWS development
– Repl development
– Javascript-es6-npm-mocha

I’m already doing docker today!

Books to Believe in

Over the past year these 28 books have changed my life in fantastic ways

What A Week

In planning a future week I realized that right now my calendar is a good reflection of my priorities and how I ideally would like to spend much of my time.  Usually I have a lot more standard and recurring meetings, so this is really a rare week that works out this way.

• Running a workshop
• 1:1 with directs plus my manager plus others
• Presenting on Ruby
• Working on Docker with the SIT team
• Leading a book club
• Quarterly Town Hall
• Quality Foundation Initiative meeting

There’s nothing proprietary ( i covered up 1 mtg details) so here it is

 

Obstacle Course Weekend

But not the physical kind.

I decided to play around with using Ruby for minimal testing setup, a current interest of mine – basically trying to reduce dependencies in a world where we want to spin up vm images such as docker in seconds all day long and the less to install the better for many reasons.

The obstacles I run into were not unusual to me at this point, but the ‘newbie person’ within me could clearly see that a newbie would be so lost with this stuff.

What happened:

1) I created a Gemfile to make sure my dependencies are installed.  Pretty minimal:

$ cat Gemfile
source 'https://rubygems.org'
gem 'minitest'
gem 'rspec-expectations'
gem 'selenium-webdriver'

Then I bundle to install them and got this:

$ bundle
Fetching gem metadata from https://rubygems.org/.........
Fetching version metadata from https://rubygems.org/..
Resolving dependencies...
Installing ffi 1.9.25 with native extensions

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

current directory: /tmp/bundler20180805-19518-uwhdxgffi-1.9.25/gems/ffi-1.9.25/ext/ffi_c
/usr/bin/ruby2.3 -r ./siteconf20180805-19518-1pqqp5e.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h

extconf failed, exit code 1

Gem files will remain installed in /tmp/bundler20180805-19518-uwhdxgffi-1.9.25/gems/ffi-1.9.25 for inspection.
Results logged to /tmp/bundler20180805-19518-uwhdxgffi-1.9.25/extensions/x86_64-linux/2.3.0/ffi-1.9.25/gem_make.out
Using diff-lcs 1.3
...
An error occurred while installing ffi (1.9.25), and Bundler cannot continue.
Make sure that `gem install ffi -v '1.9.25'` succeeds before bundling.

Which is pretty cryptic !  At first it just seemed like some issue with ffi
What caught my eye though was the “mkmf.b” error.  That seemed odd.  Googling that showed that some folks needed ruby-dev so (on Ubuntu) I did

sudo apt-get install ruby-dev

and then bundle again and this time the webdriver dependency on ffi was able to be resolved.

I also noticed my sudo prompt kept saying

unable to resolve host my-machine-name

and I found that was because the /etc/hosts file had the old machine name before I changed it.  So I fixed that while I was at it too (required sudo vi /etc/hosts)

Not Rocket Science and resolved in 5 mins but this is the sort of thing that just trips so many folks up – including myself before I went through this sort of thing regularly and just got used to it.

Good vs. Bad

Which would you rather maintain?

 

Functional creepin’ in?

Practicing the craft (programming).  Couldn’t stop messing with this one.
Kinda proud
– max 1 param on methods!
– 5 line max on methods !
– 50 line class
– readable code

But what’s what all the Proc’s and map operations (selects)?
Seems like I’m starting to get that functional feelin’!

class FerryBookings
  attr_accessor :current_side_of_river
  attr_accessor :number_of_ferry_trips
  attr_accessor :passengers

  def initialize
    @passengers = []
    @current_side_of_river = 'N'
    @number_of_ferry_trips = 0
  end

  def add_passenger(direction_headed)
    passenger = Passenger.new(direction_headed)
    @passengers << passenger
  end

  def ferry_person(person)
    person.moved = true
    switch_river_side
  end

  def ferry_all_passengers
    not_moved = Proc.new { |p| !p.moved? }
    while @passengers.select(&not_moved).size > 0
      passengers_left= @passengers.select(&not_moved)
      make_trip_south_and_north(passengers_left)
    end
  end

  def make_trip_south_and_north(passengers)
    ['S','N'].each do |direction|
      by_direction = Proc.new { |p| p.direction_headed == direction }
      ferry_trip(passengers.select(&by_direction))
    end
  end

  def ferry_trip(passengers)
    if passengers.count > 0
      ferry_person( passengers.first )
    elsif passengers_left_to_move?
      @number_of_ferry_trips += 1
    end
  end

  def passengers_left_to_move?
    @passengers.select{ |passenger| !passenger.moved? }.size > 0
  end

  def switch_river_side
    @current_side_of_river = (@current_side_of_river == 'N') ? 'S' : 'N'
    @number_of_ferry_trips += 1
  end

end

 

Code Review Breakfast, June 2018

payoff time

I’ve spent the last 9 months leading, training, cajoling, inspiring and refining 30 new software engineers during their first major software engineering jobs.

This has been an amazing learning experience and has led me to a whole new appreciation of how much value I can add as 1 person vs. the value I can add in helping a large number of engineers achieve quality in their job.

I’ve also received a lot of positive feedback from these engineers and others around me and I am humbled by how many folks have told me how much they admire my passion for quality which is behind everything I do.

I’ve been honored to see these new engineers grow fantastically in their first few months.  I’ve treated them with respect and humility and I’ve put their growth and success as my main goal.  It’s paid off fantastically.

I’ve focused on Total Candor so I can be a kick-ass boss which happens to also be the name of a great book I recently read .  Other recent great reads have included:

• Powerful: Building a Culture of Freedom and Responsibility
• Metaprogramming Ruby 2: Program like the pros
• The Managers Path: A Guide for Tech Leaders navigating growth and change
• The Clean Coder: A Code of Conduct for Professional Programmers
• Managing the Unmanageable: Rules, Tools and Insights for Managing Software People and Teams

I am now even more charged to chart the right course and do the right things!

Great Books for this months reading

Good Advice from Great Programmers

Fix the hard problems first – Joe Armstrong

When I’m just writing the first version of a program, I tend to put everything in one file. And then I start seeing structure in that file. Like there’s this block of things that are pretty similar… and the API sorts of builds up organically that way. The design process is definitely on an on-going thing; you never know what the design is until the program is done. So I prefer to get my feet wet as soon as possible; get something on the screen so I can look at it sideways. – Zawinski

Not knowing something doesn’t mean you’re dumb – it just means you don’t know it yet – Zawinski

It doesn’t matter how good you are; you can’t get an API right until you’ve tried to code to it – Joshua Bloch

How to Debug? print statements – multiple authors.

I’m really quick on the trigger for throwing stuff out – Ken Thompson

That was how I learned programming – basically taking one program that I made up myself and sitting at a machine over a period of some weeks, and kept getting it to work a little better and a littler better – Robert Knuth

So if I can get my kicks out of finding errors then I just have to make sure that I forget that I was the author of the program. I try to imagine that someone else was the author – Robert Knuth

Organizational Change

When a person or an organization wishes to change, one of the hardest parts of that change can be allowing yourself to accept what others see in you and your organization.

Command and Control is a good example of this.

Using the term “Command and Control” and categorizing ones current organization as that is a very risky and provocative approach.  It is almost guaranteed to annoy folks who will frequently reject the categorization as unfair and inaccurate and feel that it is missing the point and missing the value of current processes within in the organization that have been learned over time and through hard lessons.

Here’s a different approach to identifying command and control:

When a person is asked for information and the response is “sure. I will ask [other people] about that and get back to you” this is a different indicator, but a clear indicator non the less, that command and control is at play.  When the person who is initially asked for the connection remains in the loop they are seeking to validate their position and ego.  This is an uncomfortable fact that I have seen in every organization and industry I have worked in over the past 30 years.  It is defended in many ways as reasonable, practical and simply common sense.  Modern and re-invented organizations have learned however that this doesn’t work well at all in today modern, agile, flexible world.

In many agile organizations, different approaches are taken when this situation (a person seeks information) comes up:

• The person who knows the other people connects the two third parties to each other and then _steps back and removes themselves_ from the current and future  conversation.
• All relevant parties are invited to a custom conversation (for example a slack channel) created on-the-fly just for the directly involved people.  The ‘person in the middle’ may be one of the members but again they largely ‘step back’ and allow the other members to communicate directly for the current and future conversation.

Waive your moral rights ?

https://interviewing.io/terms

“To the extent permitted by applicable laws, you hereby waive any moral rights you have in any User Content.”

I understand legal rights but waiving my moral rights?

really ?

Tech Traveler Ten Tips

– Join your airlines frequent flyer program
– Join your hotels frequent visitor program
– Install the hotel mobile app on your phone
– Install the airlines mobile app on your phone
– Enable your phone as the room key if available
– Have and charge a powercell for device charging
– Upgrade seats soon after booking instead of at check-in
– Use airplane mode anywhere when you need to save battery power
– Avoid “basic economy” tickets on orbitz, expedia, etc. – no upgrades
– Buy wifi access ahead of time and be sure your login works before flying

Auto Complete for irb with tab

How could I not have known about this for 10 years !

in

.irbrc

require 'irb/completion'

create new github repo from command line

https://coderwall.com/p/mnwcog/create-new-github-repo-from-command-line

or, as I did at the command line:

$ curl -u durrantm https://api.github.com/user/repos -d “{\”name\”:\”mastering-docker\”}”

 

 

Docker Time

Installing on my Mac and learning via

https://www.pluralsight.com/courses/docker-deep-dive

 

1 Week, 41 commits, 40 refactors

eXtremely clean code

40 refactoring changes, 1 functional change – and even that included a refactor!

 

Building my own cloud testing pipeline

This used to be hard.  Take weeks and months.  Now the only hard piece is believing just how easy it is.  I did the following in a matter of minutes.

• Set up a new git hub project in my account (1 minute)
• Added some UI tests I wrote last week (they took me 2 weeks to write).
• Created a Jira+Confluence account & project and wired them into Github (1 minute)
• Add content to the CMS (confluence) (5 minutes)
• Created a Slack account and wired its notifications into Github (3 minutes)
• Created a CircleCI account and wired its notifications into Github (10 minutes)

So now I can:

Have a master document in Confluence which refers to tickets which are links to…

^ Jira tickets which you can click on and immediately click through to see commits

^which you can click through to see

^ Github Change Requests which you can learn about because when they are changed by developers you get…

^ Jobs submitted to CircleCI which have…

^ Slack notifications in public channels

With most of the integration work boiling down to:

Agile Transformation and Management

Many companies have gone or are undergoing Agile transformations.
There is typically a large focus on the dramatic change for regular ‘workers’ in how they do their daily activities.

One item that can easily be over-looked however is the change in daily practices for Management.  Some new organizations have eliminated all ‘supervisor’ and ‘manager’ roles, just leaving developers and a director.  Other, existing organizations that are transforming to Agile have difficult decisions to make about how to retool their management.

Failing to recognize and change can lead to management continuing to use waterfall approaches and not understanding the new empowerment model for workers.

Some of the mis-matched approaches seen in this area are:

• Using personal checklists to make sure a project is on track
• Using personal relationships to get non-planned work done
• Adding ‘important and urgent’ tasks without going through the full process
• Praising off-board work as heroic
• Discussions on team effectiveness without the team present
• Assigning or changing team resources without full team involvement
• Using traditional project management skills to try and manage the Agile process

Tech Traveler Tips

Tips for Tech for Frequent Travelers

• Multiple ipods if used
• Audio cable in every bag
• Install airline mobile apps
• Charge check the night before
• Volutz USB cables in every bag
• Install travel booking mobile app
• Extra laptop charger just for travel
• Kindle for all books and magazines
• Clear cosmetic bag for cable storage
• Battery for mobile device charge ups
• Bluetooth noise cancelling headphones
• USB wall power plug adapter in every bag
• Check comfort seat upgrades 2 weeks ahead
• Use Airplane Mode anywhere to save battery life

Hello Keep, Goodbye Evernote

I wondered if Google has something for notes, so I searched and found Keep.

bye bye evernote

oh and “btw” worried about that MS paint news? No fears.  You can ‘add drawing’ and do that too:

The Ultimate Quality Challenge

The Ultimate Quality Challenge has become clear to me.
Here’s some of the things that I’ve found it is NOT about:

• Programming
• User Experience
• Standards and Guidelines
• Agile Workflows
• User Acceptance Testing

Don’t get me wrong, all the above are critical to Quality Engineering.
However I am increasingly finding that the Ultimate Quality Challenge is one thing – hiring competent and passionate quality engineers.  This is a prerequisite to all of the above quality activities – without actual people, nothing is implemented (even automation!)

Hiring engineers in the quality space is very challenging for a number of reasons:

• Historically QE was QA and meant manual testing, typically a lower job grade
• Quality Engineering now means automation with different skill requirements
• The majority of engineers like to work with application code, not tests only
• Quality Engineers may suffer from several ‘second class citizen’ issues
• Compensation may lead the best and brightest to switch to app dev roles
• Peer respect may lead some to switch to app dev roles as they become skilled
• Availability of QE Engineers for hiring is extremely limited (2017)
• Current automation engineering differs from new javascript practices
• Quality Engineering is often not championed as a cool and useful thing

I want to help change that!
Some of the things I am doing:

• Active on sqa.stackexchange.com asking and answering questions to promote good quality practices.  I am #1 world-wide in 2017 !
• Active in Ministry of Testing slack community
• Presenter on Quality at work, local meetups, national and international conferences

ES6: Destructuring, Object Literal Enhancement and the Spread operator

Destructuring

Scoping an object locally

From an object

    var {pref1, pref2} = user  // user is object w/ keys pref1, pref2

From a functions arguments

    var function_x = ({firstname}) => {
      full_name = `${firstname} of boston`
    }

From an array

var [first_town] = ['Boston', 'Salem', 'Ham']  // 1st_town= Boston
var [,,last_town] = ['Boston', 'Salem', 'Ham']  // lst_town= Ham

Object Literal Enhancement

I’m guessing that name is going to change…
The opposite of destructuring.
Puts objects and methods together.

Objects from variables

var shell='bread'
var filling='butter'
var sandwich = {shell, filling}  
// New object 'sandwich' with contents
// {shell: 'bread', filling: 'butter'}

Object methods from variables

// Method is 'toast'
var toast = function() { console.log('Toast it for {time}') }
var toastedSandwich = {shell, filling, time}
toastedSandwich.toast

Spread Operator “...“

Array values

The actual values from an array (not the array itself)

var arr1 = ['Buick', 'Ford']
var arr2= ['Edsel', 'Dodge']
var arr3 = [...arr1, ...arr2]

Function argument values as an array

function direction(...args) {}

Object Values

The key-value attributes from an object, not the object itself, e.g.

obj1={a:1, b:2}
obj2={c:3, d:4}
obj3={...obj1, ...obj2}

 

Essential Books

If you are an application or automation developer in the Ruby space I highly recommend these (selected from several hundred books)

HTTP, GET, POST and the REST

Enjoying this once again.

A great way to really understand what’s going on in your browser

If QE’s can aborb most of this (it’s dense – be warned) they’ll be in a good spot with devs!
I suggest a chapter a day (=13 days)

Automated Link Checker

Broken links can affect your ranking with Search Engines

LinkChecker – https://wummel.github.io/linkchecker/ – is a free, GPL licensed website validator. LinkChecker checks links in web documents or full websites. It runs on Python 2.x

Includes both gui client and command line option, e.g.

linkchecker http://www.example.org/myhomepage/

More Javascript for testers

Objects have prototype methods which are shared for all instances

Useful for methods that apply to all instances

Person.prototype.some_method_name=  function() {
  commands;
};

Avoid for properties that store state data, e.g. arrays with elements managed by various methods.

When you need to define multiple methods on the prototype you can use the format below, however they should also define the constructor as shown so that the prototype is Person and not just Object:

Person.prototype = {
  constructor: Person,
  first_method: function() {...},
  second_method: function() {...};  
};

You can add methods to prototypes for all objects including builtins objects such as Array, String, etc.

Inheritance is through above Prototype Chain, e.g.

var instance2 = Object.create(instance1, {
  name: {
    value: "some_new_thing",
    ...
    }
});

next

IIFE – Immediately Invoked Function Expression

Commonly used in javascript for modules.
Note surrounding parens and parens at end

Module Pattern

var person = (function() {
  var age = 25;  // This is private
  return { ... } // These are closures and are public 
}());

Revealing Module Pattern

var person = (function() {
  var age = 25;
  function x() {...}
  return { x: x, ...}; // Note same name (x)
}());

Private Members use ‘this’

  this.getAge = function () { return age; };

 

Javascript Goodies

Functions

function declarations

function thing(p1,p2) {
  code
  return value;
}

• Starts with word ‘function’
• Not separately assigned to a variable
• WILL be hoisted and declared at top

vs

function expressions

var thing = function(p1, p2) {
  code
  return value;
};

• Assigns function with no name (annoymous) to a named variable
• Will NOT be hoisted to top, will be defined inline

IIFE’s – immediately invoked function expressions

(function(){ /* code */ }()); // Like this
(function(){ /* code */ })(); // or this

– key identifier is surrounding parens – ()

– invoked at actual runtime

Function Overloading

Achieved with functions by examing arguments, e.g.

function abc() {
   if (arguments.length === 0) {     this code; }   
else {
     other_code;   } }

The three ways to use ‘this’ with methods in javascript functions

1 With call

some_function.call(this, "text")

A particular value for ‘this’ and specific parameters

2. With apply

some_function.apply(this, "text")

A particular value for ‘this’ and, specifically, an array for any parameters

3. With bind

some_function.bind(this, "text")

‘this’ is the value for the new function, the rest are named parameters for ‘new’

Object Properties

Common Internal Properties

• Extensible
• Prototype

Data Properties, i.e. values

All Data Properties also have two attribute methods:

• value
• writeable

or

Accessor Properties, i.e. functions

All Accessor Properties also have two attribute methods:

• get
• set

  e.g.

var tree = {
  _name: "Oak",
  get name() {
    console.log("here1");
    return this._name;
    },
  set name(value) {
    console.log('here2'); 
    this._name = value;
  }
};

Retrieving Property Attributes

Object.getOwnPropertyDescriptor()

Constructors and Prototypes

Constructors use ‘new’ to create a new instance of a method.

Functions should be capitalized when they are used as constructors – you will be creating new instances of them with ‘new’.

All functions have an internal prototype property
You can check for it with object.hasOwnProperty(name)

Rubyists may find its usage has some similarities to Ruby’s class level methods.

When an object (function w/new constructor) has methods that are shared by all instances use a prototype function for the Object

YourObject.prototype.yourFunctionName = function() {...}

When an object (function w/new constructor) has attribute values that are shared by all instances use a prototype attribute for the Object

YourObject.prototype.yourVariableName = value