Reading notes: UI Testing with Puppeteer

Preface

• Puppeteer is
  • A browser automation tool
  • Created by google
• Used for
  • Web scraping
  • Task automation
  • Content generation
  • Web monitoring
  • UI testing
• Covers
  • Primary UI testing
  • Also web scraping, content generation
  • Entire puppeteer API
• For
  • QA
  • Developer
• Code
  • GitHub - PacktPublishing/UI-Testing-with-Puppeteer: UI Testing with Puppeteer, published by Packt

Chapter 1: Getting started with Puppeteer

What is browser automation?

• Mac app: 'Automator'
• Automation in the cloud
• Mail rules
• browser automation is telling an app to do a repetitive task in the browser for us.
• Automation by API
• 2 apps to automate: browser and our web app
• Selenium and Puppeteer
  • Differences
    • Selenium: cross-language, cross-browser
    • Puppeteer: for chromium
    • Interface between the tool and the browser
    • The interface between the tool and the user
  • Selenium's approach
    • WebDriver spec, testing and simplicity
  • Puppeteer's approach
    • grabbing all the developer tools that Chromium has and making them available to the user
    • More tools, more power
    • Created in javascript
    • takes advantage of the headless capability of Chromium

Introducing Headless browsers

• Headless browsers
  • A headless browser is a browser that you can launch and interact with using a particular protocol over a particular communication transport, with no UI involved
  • Who use chromium
    • Chrome
    • Edge
    • Opera
    • Brave
  • The ie and safari have no headless
  • There are webkit builds support headless (playwright)
  • Start headless
    • Chrome
      • ~ % <chrome-app-path> --headless --remote-debugging-port=9222 --crash-dumps-dir=/tmp
    • Firefox
      • ~ % <firefox-app-path> --headless
    • Both use the Chromium DevTools Protocol.
  • The Chromium DevTools Protocol
    • F12 dev tool use it too
    • Puppeteer for friendly encapsulate
• Introducing Puppeteer
  • Puppeteer is nothing more, and nothing less, than a Node.js package that knows how to open a browser, send commands, and react to messages coming from that browser
  • supports Chromium and Firefox
  • Other languages
    • Puppeteer-Sharp for .net
    • Pyppeteer for python
  • 

• 
  • Browser
    • 'Connect' to browsers
      • launched by Puppeteer
      • is already running on your local machine
      • browser running in the cloud (Browserless.io)
  • Browser context
    • Browser session, not browser window, think Incognito Mode
  • Page
  • Frame
    • At most one main frame
  • Worker
    • Do not talk about
  • Execution context
    • a mechanism Chromium uses to isolate the page from the browser extensions
    • Each frame will have its own execution context.

Puppeteer use cases

• Task automation
  • download a report
  • fill in a form
  • check your website's health
• Web Scraping
  • Web Scraping has a reputation for being illegal
• Content generation
  • Screenshots
  • PDF files
• End-to-end testing
  • Maybe selenium is better (selenium grid)

Setting up the environment

• Node.js
  • Ver > 12.18.3
• Visual Studio Code

Our first Puppeteer code

• 

• 

• 
• Installing Puppeteer
• 
  • Puppeteer follow semver
  • Download a specific version of Chromium
• Hello world in Puppeteer
  • Nav to a page and take a screenshot
  • 

Asynchronous programming in JavaScript

• I know a lot about this

Chapter 2: Automated Testing and Test runners

• UI Testing vs. End-to-End testing

Technical requirements

• Code at chapter2 of repo

Introduction to Automated Testing

• Testing is a fundamental task in software development
• Who involved in the testing
  • Backend developers
  • Frontend developers
  • QA
  • Managers (product and project)
  • Different types of test
  • 
  • Frontend developer should
    • Not limit to UI
    • Be able to write unit tests
    • And service tests
• Unit tests
  • More unit tests, less UI tests
  • Need to be fast and isolated
  • What roles use it?
    • Backend developers
      • TDD process
    • Frontend developers
      • Impossible in the past
      • should be able to use Puppeteer to write UI unit tests
    • QA
      • Not involved yet
    • Managers
      • Managers won't write unit tests, but they need to know the importance of writing unit tests and investing time in them.
  • 4 benefits
    • Show how your code works
      • Read it first when review code
      • Check if complete
    • Make refactoring possible
      • I don't know what it should be without unit tests
    • Prevent regressions
      • Regression is an involuntary change
      • Changes in unit tests are a red flag, pay attention to it
• Service tests
  • Also know as Integration Tests
  • These tests will check how your code interacts with other components
  • Slower and less stable, because more thing involved
  • If you are a frontend developer, this is where you would invest most of your time
  • What roles use Integration tests?
    • Backend
    • Frontend
    • QA
      • Same as backend and frontend, but different perspective
      • Visual regression test
        • Baseline images
        • Change code
        • Another image
        • Compare
    • Manager
      • Provide time and tool
      • What integrations are
• End-to-end tests
  • Goal: application works as expected through the entire workflow
  • More workflow require more e2e tests
  • Roles
    • QA
      • This is the QA Analyst's land
      • Developers play an important role, helping the QA team to do their job efficiently, leave hints

Test runner features

• You can write test without test runner, but not scale
• Features
  • Easy to learn and run
  • Group tests by functionality
  • Ignore tests if needed
  • Run only one test
  • Assertions
    • Value equal
    • Null or not null
    • Contains a value
    • Expected something to fail
  • Tools to set up and clean up the environment
  • Reports

Available test runners

• Jest
  • By facebook
  • Have all the features
  • Has snapshot tool: json dom
  • Plugins, like jest-puppeteer
  • Visual regression test package: 'jest-image-snapshot'
• Mocha
  • Puppeteer team use it
  • Have all the features
  • Plugins: mocha-puppeteer, mocha-snapshots
  • Mocha + Chai (assertion lib)
• Others
  • Jasmine
  • Karma
  • AVA
• We use Mocha + Chai

Creating our first test project

• Install packages
• 
• First demo: test the page title
• One or many 'describe' per file
• Mocha use 'test' folder by default
• file: home.tests.js
• 
• Require things from 'chai', no mocha
• Code test
• 
• Use test hook
• 

• 

• 

• Fire and forget
• 
• Tests
• 
• Setup 'test' script
• 
• 2000ms timeout error
• 
• Change timeout to 30s
• 
• Result report
• 
• Use list reproter
• 

Organizing our code

• Organize code is important, or every one lose
• POM: Page Object Model
• HomePageModel
• 
• Pass in a page and implements
• 
• Put to folder /test/POM/HomePageModel.js
• Use it
• 
• Replace previous code
• 
• Problem: hard-coded values
• Solution: config.js
• 
• Final
• 

• 

Chapter 3: Navigating through a website

Technical requirements

• code
  • GitHub - PacktPublishing/UI-Testing-with-Puppeteer: UI Testing with Puppeteer, published by Packt
  • Base path: chapter3
  • 

• 

• 2 terminals
  • One for website
  • One for tester
    • In the 'init' folder
    • $ npm run test

Creating a puppeteer browser

• Using the Puppeteer.launch function
  • Launch options
    • product: Which browser to launch. At this time, this is either chrome or firefox.
    • ignoreHTTPSErrors: Whether to ignore HTTPS errors during navigation.
    • headless: Whether to run the browser in headless mode. Defaults to true unless the devtools option is true.
    • executablePath: Path to a browser executable to run instead of the bundled Chromium.
    • slowMo: Slows down Puppeteer operations by the specified number of milliseconds. Useful so that you can see what is going on.
    • defaultViewport: Sets a consistent viewport for each page. Defaults to an 800x600 viewport. null disables the default viewport. A viewport is an object with the following properties:

a) width: page width in pixels.

b) height: page height in pixels.

c) deviceScaleFactor: Specify device scale factor.

d) isMobile: Whether the meta viewport tag is taken into account.

e) hasTouch: Specifies whether the viewport supports touch events.

f) isLandscape: Specifies whether the viewport is in landscape mode.

• args: Additional arguments to pass to the browser instance.
• ignoreDefaultArgs: If true, then do not use puppeteer.defaultArgs(). If an array is given, then filter out the given default arguments.
• handleSIGINT: Close the browser process on Ctrl +C.
• handleSIGTERM: Close the browser process on SIGTERM.
• handleSIGHUP: Close the browser process on SIGHUP.
• timeout: Maximum time in milliseconds to wait for the browser instance to start. Defaults to 30000 (30 seconds). Passing 0 disables the timeout.
• dumpio: Whether to pipe the browser process stdout and stderr into process.stdout and process.stderr.
• userDataDir: Path to a user data directory.
• env: Specify environment variables that will be visible to the browser.
• devtools: Whether to auto-open a DevTools panel for each tab. If this option is true, the headless option will be set to false.
• pipe: Connects to the browser over a pipe instead of a WebSocket. Defaults to false.
• extraPrefsFirefox: Additional preferences that can be passed to Firefox.

• Option.headless
  • 
  • you will get that banner saying that "Chrome is being controlled by automated test software."
  • you can't remove that banner
• Option.userDataDir
  • Default: new dir on launch, delete it when close
  • Use it for checking local stores (cookie..)
• Option.executablePath
  • Use your own browsers, set userDataDir to the same dir
  • You can use puppeteer-core instead of Puppeteer
• Option.defaultViewport
  • Default to 800 * 600
  • Breakpoints to use
    • 
  • Consider breakpoints when testing
• Option.product
  • Puppeteer work with firefox (experiment)
  • Install for firefox support
  • 

• 

• Options.args
  • A array over 1400 flags
  • --no-sandbox
    • You might need to create a user with the right permissions to use Puppeteer in a more restricted context
    • --window-size
    • --proxy-server, --proxy-bypass-list
    • Option.extraPrefsFirefox for firefox config
• Mobile options
  • deviceScaleFactor, isMobile, hasTouch, and isLandscape will help us set up mobile emulation.

• Options in practice
  • Place launch options in config file
  • Test persisted login
    • 

Navigating through a site

• 4 ways to nav
  • Type url, bookmark
  • Go back, forward, reload
  • Click element
  • Browser redirect

• Options of page.goto(url, options)
  • 
  • Timeout
    • Nav timeout, generic timeout
    • How to do
      • Set timeout in config file
      • Call (no async):
        • page.setDefaultTimeout(timeout)
        • page.setDefaultNavigationTimeout(timeout).
    • Default to 30s
  • waitUntil
    • Why: page.goto() resolve when command sent, but no a full page
    • Default to 'load': load event fired, all is loaded (css, img..)
    • 'domcontentloaded': DOMContentLoaded event fired, html to DOM only
  • 'networkidle0', 'networkidel2'
    • In the past 500ms
  • Referrer
    • What page is requesting resouces
    • Called 'referer' due to typo in http spec
  • Other nav ways, same options
    • 
    • No referer option
  • The page.goto() will return response object

Using the response object

• Getting the response URL
  • Useful for redirecting
• Getting the response status code
  • 1xx informational response
    • the request was received, continuing process
  • 2xx successful
    • the request was successfully received, understood, and accepted
  • 3xx redirection
    • further action needs to be taken in order to complete the request.
    • 301 – Permanent Redirect and 302 –Temporary Redirect.
  • 4xx client error
    • the request contains bad syntax or cannot be fulfilled.
    • 401 – Unauthorized and 403 – Forbidden
  • 5xx server error:
    • the server failed to fulfill an apparently valid request.
    • 500, 503
  • Get status code
    • response.status()
    • response.ok().
• test: "The admin page should redirect you to the login page"
  • 
• Redirect chain
  • 'response.url()'
  • 'reponse.request()'
  • request contains the list of all the redirections a request went through
  • 

• 

Introduction to continuous integration

• what continuous integration (CI) is about
  • Wouldn't it be great if a tool would guarantee that not a single line of code would break the functionality you are testing?
  • Popular Cis
    • 
  • Github action demo
    • Create a new repo
    • 
    • Set up the CI
      • Yaml files inside the .github/workflows/
      • Steps
      • 
      • Test.yaml
      • 

 

Chapter 4: Interacting with a page

• Arrange-Act-Assert (AAA)
  • 
• Dev team improve html for easily find elements
• 2 sets of apis
  • Action functions
  • Emulation functions
• Element handle

Technical requirements

• Repo/chapter4
• On chapter4: npm install
• On Chapter4/vuejs-firebase-shopping-cart: npm install

Introduction to html, the dom, and css

• All web techs go to html, css, javascript
• Firefox tool: View | Page style | No style
• Html not restricted as xml
• Broken html maybe rendering, but avoid it
• Semantic elements vs. div
• 22 types for input
• Select + option
• Modern ui make testing challengin (fake input)
• Data-xx for custom attrs
• Aria attrs
• CSS selectors for locating elements
• Why need to know
  • You will spend half of your time inside the developer tools, and the most frequent element in your code will be a CSS selector.

Finding elements

• Page.$ for query one
• Page.$$ for query many
• Don't use style class for locate elements
• Login page
  • Email input: #email
  • Password input: #password
  • Login button: #login-form [type=submit]
• Home page
  • Dev teams can use data-test-xxx to help testing
  • 

Finding elements using xpath

• Dev tool util functions: $x('//*')
• //.
  • //div//a == div a
• /.
  • /div//a == ::root > div a
  • Root is doc, not html
• *
  • //* == *
• [@attr=value]
  • //input[@type=button] == input[type=button]
• [text()=value]
  • //div[text()='hello']
• [contains(text(), value)]
  • //*[contains(text(), 'grid')]
• Complex one
  • 
  • Select parent
    • $x('//div[@class="row"]/p[1][number(substring-after(text(), "$")) > 2000]/../..')
• The puppeteer way
  • Page.$x()
  • We want to test that the Macbook Pro 13.3' Retina MF841LL/A has 15 items left in stock, and the price is $1,199.
  • 
  • $, $$, $x return 'element handles'

Interacting with elements

• Login page, enter email, password, click login button
• Typing on input elements
  • 'eh' for element handle
  • Eh.type(text, [options])
  • Options.delay: delay between letters
  • 
• Clicking on elements
  • Eh.click([options])
  • Options
    • Button: left | right | middle
    • clickCount: default to 1
    • Delay: between mouse down and up
  • 
• Selecting options in drop-down lists
  • switch between the grid and the list view
  • Eh.select(…values)
    • Value is value for options, not text
  • 

• 

Keyboard and mouse emulation

• Keyboard shortcut eh.press(key, options)
  • For down(key, options) and up(key)
  • Options
    • Text: what text will be typed
    • Delay: down to up
  • Example: remove a word
  • 

• Example: search on github
• 
• Mouse move
  • Eh.mouse.move(x, y, options)
    • Options.steps: how many move events fired, default to 1
  • With keyboard: up, down, press
  • With mouse: up, down, click
• Mouse wheel
  • Eh.mouse.wheel(optons)
  • Options
    • deltaX: how far to scroll x
    • deltaY
• Chap4/tictactoe.html
• 
• Mouse Hover
• 

• El.boundingBox()
• Can also ' await eh.hover() '

Debugging tests with visual studio code

• Javascript debug terminal
• 
• /.vscode/launch.json
• 

Chapter 5: Waiting for elements and network calls

• Internet, the web
  • 90s vs. 2000s
  • Gmail show the true web
  • Have to wait, so the flaky tests
    • Waste of time
    • Flase alarm
    • Hurt reputation
    • In the end, you lose, so your team

Technical requirement

• Code: /repo/chap5

Waiting for page to load

• The option.waitUntil
  • Done loaded dom: domcontentloaded
  • Done loaded all: load
  • No connection in past 500ms: networkidle0
  • No > 2 connection in past 500ms: networkidle2
• Case study
  • Full site
  • 
  • Dom content loaded
  • 
  • Loaded
  • 

• 

• Network idle0 vs. idle2
  • Can we just wait for 10 seconds? No, too long time.
  • According to what we need, find a balance, maybe domcontentload is enough
  • Set waitUtil could be useful, but often not enough

Waiting for elements

• Before Act on element
  • It is in dom
  • It is visible
• How to wait for popup?
• 2 helpers functions
  • ' waitForSelector(selector, options)
  • ' waitForXPath(xpath, options)
  • Options
    • .timeout: default to page.setDefaultTimeout() > 30s
    • .visible: default to false
    • .hidden: loading indicator
  • Return
    • An element handle
    • Null for not found, may be hidden (except hidden = true)
  • 
  • Why requery?

Waiting for networking calls

• Page.waitForResponse(urlOrPredicate, [options])
  • 'urlOrPredicate': url or a func check response
  • Only option, timeout
• Arrange, Act, Await
  • Race condition
    • 
    • Need no asset, done is good enough
    • 
    • Make waitForResponse earlier, when wait after happened, you got timeout
• Page.waitForRequest(urlOrPredicate, options)
• Fire and forget
  • Call a func returns a promise, but do not wait it
  • Like military missile
  • 
  • If goto fails, the waitForRequest will fail, and the test will fail.
  • Page.waitForNavigation([options])
    • Eg. Search then redirect
    • 

Waiting for page events

• waitForRequest and waitForResponse are just wrappers around the request and response events the page offers.
• 
• Remote listener
• 
• Listen once
• 
• The close event
  • For page is closed, often popup page
• The popup event
  • Open a new tab or window
  • 
• Target created event
  • 
• The console event
  • For every console output
  • Event props
    • Text()
    • Type()
    • Location()
    • Args()
• The dialog event
  • Event.type()
    • Alert
      • Confirm use event.accept()
    • Confirm
      • Event.accept()
      • Event.dismiss()
    • Prompt
      • Event.accept(input)
      • Event.dismiss()
    • The beforeunload type
      • Like confirm

The headless recorder

• A chrome extension 'headless recorder'
• Give you puppeteer and playwright code