TOC: Test Automation Using Perl

• Preface
  1. Why Test?
  2. What are automated test?
  3. Manual testing
  4. Why Automate?
  5. Functional Testing
  6. Unit/Integration/Acceptance Test
  7. White box and Black box testing
  8. Objectives of the course
  9. Plan of the seminar
  10. Process of testing
• Basic Testing Framework in Perl
  1. Things we are going to test
  2. Testing a simple Perl module
  3. Calculator test
  4. Test with expected results
  5. More test cases, more output
  6. Complex output
  7. ok, not ok
  8. Print only ok/not ok
  9. Refactor - Write the ok function
  10. Introducing Test::Simple
  11. Test::Simple when everything is ok
  12. Test::Simple - missing test
  13. Test::Simple - too many tests
  14. Add names to the tests
  15. Exercise: Write tests for fibo
  16. Exercise: Write a test to test the Anagram checker
  17. Exercise: Enlarge the test
  18. Solution: Write tests for fibo
  19. Solution: Write a test to test the Anagram checker
  20. Solution: Enlarge our test suite
  21. Refactor larger test suite
  22. Forget about your "plan", use "no_plan"
  23. use BEGIN block with Test::Simple
  24. Put the test cases in an external file
  25. Large test suite
  26. Harness
  27. Harness on success
  28. Harness on too few tests
  29. prove
  30. Packaging as people do for CPAN
  31. Makefile.PL for ExtUtils::MakeMaker
  32. Makefile.PL for Module::Install
  33. Build.PL
  34. Directories under t and prove
  35. Simple CPAN-like module
  36. Commands for CPAN release
  37. Exercise: Add tests
  38. Test::Simple
• Test::More
  1. Moving over to Test::More
  2. Test::Simple ok( trueness, name);
  3. Test::More ok( trueness, name);
  4. Test::More is( value, expected_value, name);
  5. Test::More isnt( value, not_expected_value, name);
  6. Test::More isnt undef
  7. note( message ) or diag( message );
  8. (note or diag) explain( a_variable );
  9. TODO
  10. TODO Verbose output
  11. TODO: unexpected success
  12. TODO: unexpected success (the code)
  13. MyTools with various functions
  14. like(value, qr/expected regex/, name);
  15. like(value, qr/expected regex/, name);
  16. Another example with like
  17. cmp_ok( this, op, that, name);
  18. is_deeply( complex_structure, expected_complex_structure, name);
  19. Function returning data from bug-tracker
  20. is_deeply on a hash
  21. Platform dependent tests
  22. SKIP some tests
  23. locale
  24. Stop running current test script
  25. Stop all the test scripts
  26. Exercises
  27. Test coverage using Devel::Cover
  28. Test coverage report example
  29. Declare your plan at execution time
  30. done_testing
  31. plan tests, no_testing, done_testing
  32. Compute test plan
  33. subtest with plan
  34. subtest with implicit done_testing
  35. subtest to restrict scope
  36. skip all
  37. Exercise: skip test
  38. Exercise: use coverage
  39. Test blocks (use subtest instead)
  40. Counting tests in the small blocks (use subtest instead)
• Test libraries
  1. Multiple expected values
  2. Multiple expected values revised
  3. Adding information with diag
  4. My own test functions
  5. My own test functions with Test Builder level
  6. Create a test module
  7. Test::Builder
  8. Test::Builder object is a singleton
  9. Skip on the fly
  10. Skip on the fly based on earlier tests
  11. Code emitting warnings
  12. Test::NoWarnings
  13. Test::FailWarnings
  14. Test with warnings
  15. Testing for warnings
  16. Testing for warnings - different warning
  17. Testing for warnings - missing warning
  18. Test::Warn
  19. Test for no warnings - the hard way
  20. No other warnings using Test::NoWarnings
  21. No other warnings Test::FailWarnings
  22. unexpected warnings Test::NoWarnings, Test::FailWarnings
  23. Test::Exception
  24. Exercise: improve test module
  25. Exercise: add is_max
  26. Exercise: is_between
  27. Exercise: test sum
  28. Exercise: Test::Exception
  29. Solution: is_between
  30. Solution: test sum
  31. Perl Best Practices - Perl::Critic
  32. Verify code layout
  33. Why number the test files?
  34. Test::Differences
  35. Test::Deep
  36. Test::File
  37. Test::LongString
  38. Test::Most
  39. Test::Trap
  40. Test::Fatal
  41. Test::XPath
  42. Sample script for testing Client-Server
  43. Sample script for testing Client-Server Win32
  44. Exercise for Test::Builder
  45. Exercise: Math::RPN
  46. Exercise: exceptions
  47. Solution
• Mocking
  1. What is Mocking?
  2. Test Doubles
  3. Test Doubles explained
  4. What is Monkey Patching?
  5. When is mocking useful?
  6. Mocking in various situations
  7. Application using random
  8. Mock random generator in BEGIN
  9. Mock random generator Mock::Sub
  10. Function that (also) writes to STDOUT or STDERR
  11. Capture STDOUT and STDERR in functions call
  12. Code with STDIN
  13. Mock STDIN
  14. Simple game to test
  15. Test simple game (small)
  16. Test simple game (big)
  17. Test simple game (exact)
  18. Exercise: test game
  19. Testing time-dependent module
  20. Module with operation based on date
  21. Mocking fixed absolute time
  22. Module with session timeout
  23. Mocking relative Time
  24. Mocking class
  25. Application using the class
  26. Testing the 3rd party class
  27. Testing our app using the 3rd party class
  28. Testing our app mocking the 3rd party class
  29. Mocking function of external web call
  30. Test the application end-to-end
  31. Mocked web test
  32. Mocking to reproduce error in our function
  33. Mocking exception
  34. Using MetaCPAN::Client
  35. Mocking MetaCPAN::Client
  36. Override printing functions (mocking)
  37. Monkey Patching
  38. Test STDIN
• Test::Class
  1. Simple module to test
  2. Test::Class simple example
  3. Test::Class with fixtures
  4. Test::Class run both test classes
  5. Test::Class inline
  6. Test::Class Load
• Running and reporting tests
  1. TAP - Test Anything Protocol
  2. prove
  3. Parallel testing
  4. prove --state
  5. Parse TAP from a file
  6. prove - run other executables
  7. TAP::Formatter::HTML by Steve Purkis
  8. Collecting Test reports
  9. Generating HTML reports from Archives
• BDD - Behavior Driven Development
  1. BDD Hello World
  2. BDD Demo
• Command line application
  1. bc - An arbitrary precision calculator language
  2. Normal operation
  3. Expect.pm
  4. Simple computation - adding two values
  5. Results
  6. Reduce output - turn it into a test script
  7. Output
  8. Expect and BAIL_OUT
  9. More than one test
  10. Output
  11. External test file
  12. Random regression tests
  13. Random and regression testing
  14. Random and regression testing - slight improvement
  15. Results
  16. Expect multiple values
  17. Test::Expect
  18. Capturing both STDOUT and STDERR
  19. Capturing both STDOUT and STDERR manually
  20. Capturing both STDOUT and STDERR using IPC::Run3
  21. Capture::Tiny
  22. Test::Snapshots
  23. Exercise: Expect
• Networking devices
  1. Introduction - pick the right abstraction level
  2. Socket level programming using Socket.pm
  3. Newline
  4. Socket level programming using IO::Socket
  5. Net::Telnet
  6. Net::Telnet for HTTP
  7. Net::Telnet configure VLAN
  8. ftp using Net::FTP
  9. ssh using Net::SSH
  10. LWP::Simple
  11. LWP::UserAgent
  12. WWW::Mechanize
  13. WWW::Mechanize for Google
  14. Exercise: Search on Financial Times
  15. Exercise: Compare exchange rates
  16. Exercise: Telnet or SSH to Unix machines
  17. Introduction
  18. Connect to the device
  19. Reduce timeout
  20. Exercise: Telnet
  21. Our test script
• Web Applications
  1. What can be tested ?
  2. Extreme web site testing
  3. Tools
  4. Small test HTTP server
  5. Fetching a static page
  6. Fetching a not-existing static page
  7. Checking good HTML
  8. Checking bad HTML
  9. What is this bad HTML ?
  10. HTML::Tidy and Test::HTML::Tidy
  11. Test using W3C validator
  12. Use a local copy of the W3C validator
  13. LWP::Simple and LWP
  14. WWW::Mechanize
  15. Web based Calculator with WWW::Mechanize
  16. Testing with WWW::Mechanize
  17. Test::WWW::Mechanize
  18. Login to Act using Mechanize
  19. More things to test
  20. Test without server Test::WWW::Mechanize::PSGI
  21. Test page with JavaScript
• Selenium
  1. Selenium documentation
  2. Selenium IDE
  3. Launch Selenium server
  4. Selenium DuckDuckGo
  5. Selenium using Chrome
  6. Selenium DuckDuckGo Search
  7. Selenium DuckDuckGo Test
  8. Selenium locator
  9. Selenium locator example
  10. Selenium content
  11. Selenium in the calc example
  12. Selenium examples with JavaScript
  13. Selenium examples with Ajax
  14. WWW::Mechanize::PhantomJS
  15. Testing Dancer
  16. Exercies: MetaCPAN
  17. Exercise: blogs.perl.org
  18. Exercise: Testing Smolder
  19. Exercise: Act
• Servers
  1. Net::Server
  2. Skeleton Server
  3. Simple Echo Server
  4. Echo Server
  5. Complex network servers
• Testing networking devices
  1. Elements
  2. Network testing
  3. Hardware setup
  4. Access the administrative interface
  5. CLI testing
  6. Configure devices on all sides of our box
  7. Run tests
  8. Check results
  9. Expect.pm
  10. Networking
  11. Network devices
  12. Devices connected to Serial or Parallel port
  13. X10 protocol
• Database Testing
  1. Database testing
  2. A couple of tools
  3. Test::More and DBI
  4. Test::DatabaseRow
  5. Test::DatabaseRow fail
  6. Test::DatabaseRow tests more than one row
  7. Test::DatabaseRow tests more than one row - failure
  8. Test::DatabaseRow without SQL
• Database access using Perl DBI
  1. Architecture of a DBI Application
  2. Create Sample Database
  3. Connect to database
  4. SELECT with one result
  5. SELECT with more results
  6. SELECT, prepare with placeholders
  7. SELECT, using hashref
  8. INSERT
  9. UPDATE
• Advanced Database access using Perl DBI
  1. Data integrity
  2. Transactions
  3. Sample database creation
  4. Disconnect from the database
  5. Attributes
  6. Error handling
  7. Debugging (Trace levels)
• Database access using DBIx::Class
  1. DBIx::Class
  2. Create Sample Database
• Microsoft Windows GUI Testing
  1. GUI testing
  2. Keyboard and mouse movements
  3. Win32::GuiTest
  4. Win32::GUIRobot
  5. Calculator
  6. Launch the application
  7. Locale
  8. Close the application
  9. Using the Keyboard
  10. Check the children
  11. Fetch content of a control
  12. Window location
  13. Move the mouse
  14. Move the cursor around the edges
  15. Close the window by a mouse click
  16. Calculate using the mouse
  17. The full calc.pl example
  18. Installing Win32::GuiTest
  19. Tools
• Internet Explorer
  1. Win32::IE::Mechanize
  2. Accessing Google
  3. Examples
• X-Windows
  1. X11
  2. X11 Tools
  3. X11::GUITest
  4. LDTP - Linux Desktop Testing Project
  5. Manual steps
  6. X11: Launch the application
  7. Let's see the coordinates
  8. Where is this top left corner ?
  9. And let's see the rest of the coordinates
  10. Smooth on the edges
  11. Closing the window
  12. Placing the cursor on one of the buttons and clicking on it
  13. So let's do all the calculations
  14. Fetch the list of Child Windows
  15. Locate the Child Windows
  16. Using the keyboard
  17. Separate the keystrokes
  18. Full solution for xcalc
• Appendixes
  1. Test related CPAN Modules
  2. Generic modules
  3. Web testing
  4. GUI testing
  5. Other sources of information
  6. Test Realistically
  7. Test Unrealistically
  8. File System testing
• Other
  1. Devel::Cover script
  2. Can module be loaded? use_ok and require_ok
  3. can_ok('Class', qw(method_a method_b));
  4. All the tests
  5. Smolder
  6. Exercise: Smolder
  7. Mocking get in LWP::Simple
  8. Read CSV file as array of hashes
  9. Sort array using sort and sort_by
  10. Range iterator
  11. Benchmark
  12. Bubble sort
  13. Insert sort
  14. Merge sort
  15. Quicksort
  16. Timsort
  17. Traverse directory tree
  18. Traverse directory tree with call back
  19. Linear search in unorderd array
  20. Binary search in sorted array
  21. Convert curl command with -F to LWP::UserAgent
  22. Modify time anomality in two files