Changes
Testing in Perl
Preface
1. Why Test?
2. What are automated test?
3. Manual testing
4. Why Automate?
5. Functional Testing
6. Unit/Integration/Acceptance Test
7. White box and Black box testing
8. Objectives of the course
9. Plan of the seminar
10. Process of testing
Basic Testing Framework in Perl
1. Things we are going to test
2. Testing a simple Perl module
3. Calculator test
4. Test with expected results
5. More test cases, more output
6. Complex output
7. ok, not ok
8. Print only ok/not ok
9. Refactor - Write the ok function
10. Introducing Test::Simple
11. Test::Simple when everything is ok
12. Test::Simple - missing test
13. Test::Simple - too many tests
14. Add names to the tests
15. Exercise: Write tests for fibo
16. Exercise: Write a test to test the Anagram checker
17. Exercise: Enlarge the test
18. Solution: Write tests for fibo
19. Solution: Write a test to test the Anagram checker
20. Solution: Enlarge our test suite
21. Refactor larger test suite
22. Forget about your "plan", use "no_plan"
23. use BEGIN block with Test::Simple
24. Put the test cases in an external file
25. Large test suite
26. Harness
27. Harness on success
28. Harness on too few tests
29. prove
30. Packaging as people do for CPAN
31. Makefile.PL for ExtUtils::MakeMaker
32. Makefile.PL for Module::Install
33. Build.PL
34. Directories under t and prove
35. Simple CPAN-like module
36. Commands for CPAN release
37. Exercise: Add tests
38. Test::Simple
Test::More
1. Moving over to Test::More
2. Test::Simple ok( trueness, name);
3. Test::More ok( trueness, name);
4. Test::More is( value, expected_value, name);
5. Test::More isnt( value, not_expected_value, name);
6. Test::More isnt undef
7. note( message ) or diag( message );
8. (note or diag) explain( a_variable );
9. TODO
10. TODO Verbose output
11. TODO: unexpected success
12. TODO: unexpected success (the code)
13. MyTools with various functions
14. like(value, qr/expected regex/, name);
15. like(value, qr/expected regex/, name);
16. Another example with like
17. cmp_ok( this, op, that, name);
18. is_deeply( complex_structure, expected_complex_structure, name);
19. Function returning data from bug-tracker
20. is_deeply on a hash
21. Platform dependent tests
22. SKIP some tests
23. locale
24. Stop running current test script
25. Stop all the test scripts
26. Exercises
27. Test coverage using Devel::Cover
28. Test coverage report example
29. Declare your plan at execution time
30. done_testing
31. plan tests, no_testing, done_testing
32. Compute test plan
33. subtest with plan
34. subtest with implicit done_testing
35. subtest to restrict scope
36. skip all
37. Exercise: skip test
38. Exercise: use coverage
39. Test blocks (use subtest instead)
40. Counting tests in the small blocks (use subtest instead)
Test libraries
1. Multiple expected values
2. Multiple expected values revised
3. Adding information with diag
4. My own test functions
5. My own test functions with Test Builder level
6. Create a test module
7. Test::Builder
8. Test::Builder object is a singleton
9. Skip on the fly
10. Skip on the fly based on earlier tests
11. Code emitting warnings
12. Test::NoWarnings
13. Test::FailWarnings
14. Test with warnings
15. Testing for warnings
16. Testing for warnings - different warning
17. Testing for warnings - missing warning
18. Test::Warn
19. Test for no warnings - the hard way
20. No other warnings using Test::NoWarnings
21. No other warnings Test::FailWarnings
22. unexpected warnings Test::NoWarnings, Test::FailWarnings
23. Test::Exception
24. Exercise: improve test module
25. Exercise: add is_max
26. Exercise: is_between
27. Exercise: test sum
28. Exercise: Test::Exception
29. Solution: is_between
30. Solution: test sum
31. Perl Best Practices - Perl::Critic
32. Verify code layout
33. Why number the test files?
34. Test::Differences
35. Test::Deep
36. Test::File
37. Test::LongString
38. Test::Most
39. Test::Trap
40. Test::Fatal
41. Test::XPath
42. Sample script for testing Client-Server
43. Sample script for testing Client-Server Win32
44. Exercise for Test::Builder
45. Exercise: Math::RPN
46. Exercise: exceptions
47. Solution
48. Devel::Cover script
49. Can module be loaded? use_ok and require_ok
50. can_ok('Class', qw(method_a method_b));
51. All the tests
52. Smolder
53. Exercise: Smolder
Mocking
1. What is Mocking?
2. Test Doubles
3. Test Doubles explained
4. What is Monkey Patching?
5. When is mocking useful?
6. Mocking in various situations
7. Application using random
8. Mock random generator in BEGIN
9. Mock random generator Mock::Sub
10. Function that (also) writes to STDOUT or STDERR
11. Capture STDOUT and STDERR in functions call
12. Code with STDIN
13. Mock STDIN
14. Simple game to test
15. Test simple game (small)
16. Test simple game (big)
17. Test simple game (exact)
18. Exercise: test game
19. Testing time-dependent module
20. Module with operation based on date
21. Mocking fixed absolute time
22. Module with session timeout
23. Mocking relative Time
24. Mocking class
25. Application using the class
26. Testing the 3rd party class
27. Testing our app using the 3rd party class
28. Testing our app mocking the 3rd party class
29. Mocking function of external web call
30. Test the application end-to-end
31. Mocked web test
32. Mocking to reproduce error in our function
33. Mocking exception
34. Using MetaCPAN::Client
35. Mocking MetaCPAN::Client
36. Override printing functions (mocking)
37. Monkey Patching
38. Test STDIN
39. Mocking get in LWP::Simple
Test::Class
1. Simple module to test
2. Test::Class simple example
3. Test::Class with fixtures
4. Test::Class run both test classes
5. Test::Class inline
6. Test::Class Load
Running and reporting tests
1. TAP - Test Anything Protocol
2. prove
3. Parallel testing
4. prove --state
5. Parse TAP from a file
6. prove - run other executables
7. TAP::Formatter::HTML by Steve Purkis
8. Collecting Test reports
9. Generating HTML reports from Archives
BDD - Behavior Driven Development
1. BDD Hello World
2. BDD Demo
Test Command line application
1. bc - An arbitrary precision calculator language
2. Normal operation
3. Expect.pm
4. Simple computation - adding two values
5. Results
6. Reduce output - turn it into a test script
7. Output
8. Expect and BAIL_OUT
9. More than one test
10. Output
11. External test file
12. Random regression tests
13. Random and regression testing
14. Random and regression testing - slight improvement
15. Results
16. Expect multiple values
17. Test::Expect
18. Capturing both STDOUT and STDERR
19. Capturing both STDOUT and STDERR manually
20. Capturing both STDOUT and STDERR using IPC::Run3
21. Capture::Tiny
22. Test::Snapshots
23. Exercise: Expect
Networking devices
1. Introduction - pick the right abstraction level
2. Socket level programming using Socket.pm
3. Newline
4. Socket level programming using IO::Socket
5. Net::Telnet
6. Net::Telnet for HTTP
7. Net::Telnet configure VLAN
8. ftp using Net::FTP
9. ssh using Net::SSH
10. LWP::Simple
11. LWP::UserAgent
12. WWW::Mechanize
13. WWW::Mechanize for Google
14. Exercise: Search on Financial Times
15. Exercise: Compare exchange rates
16. Exercise: Telnet or SSH to Unix machines
17. Introduction
18. Connect to the device
19. Reduce timeout
20. Our test script
Testing Web Applications
1. What can be tested ?
2. Extreme web site testing
3. Tools
4. Small test HTTP server
5. Fetching a static page
6. Fetching a not-existing static page
7. Checking good HTML
8. Checking bad HTML
9. What is this bad HTML ?
10. HTML::Tidy and Test::HTML::Tidy
11. Test using W3C validator
12. Use a local copy of the W3C validator
13. LWP::Simple and LWP
14. WWW::Mechanize
15. Web based Calculator with WWW::Mechanize
16. Testing with WWW::Mechanize
17. Test::WWW::Mechanize
18. Login to Act using Mechanize
19. More things to test
20. Test without server Test::WWW::Mechanize::PSGI
21. Test page with JavaScript
Selenium
1. Selenium documentation
2. Selenium IDE
3. Launch Selenium server
4. Selenium DuckDuckGo
5. Selenium using Chrome
6. Selenium DuckDuckGo Search
7. Selenium DuckDuckGo Test
8. Selenium locator
9. Selenium locator example
10. Selenium content
11. Selenium in the calc example
12. Selenium examples with JavaScript
13. Selenium examples with Ajax
14. WWW::Mechanize::PhantomJS
15. Testing Dancer
16. Exercies: MetaCPAN
17. Exercise: blogs.perl.org
18. Exercise: Testing Smolder
19. Exercise: Act
Servers
1. Net::Server
2. Skeleton Server
3. Simple Echo Server
4. Echo Server
5. Complex network servers
Testing networking devices
1. Elements
2. Network testing
3. Hardware setup
4. Access the administrative interface
5. CLI testing
6. Configure devices on all sides of our box
7. Run tests
8. Check results
9. Expect.pm
10. Networking
11. Network devices
12. Devices connected to Serial or Parallel port
13. X10 protocol
Database Testing
1. Database testing
2. A couple of tools
3. Test::More and DBI
4. Test::DatabaseRow
5. Test::DatabaseRow fail
6. Test::DatabaseRow tests more than one row
7. Test::DatabaseRow tests more than one row - failure
8. Test::DatabaseRow without SQL
Microsoft Windows GUI Testing
1. GUI testing
2. Keyboard and mouse movements
3. Win32::GuiTest
4. Win32::GUIRobot
5. Calculator
6. Launch the application
7. Locale
8. Close the application
9. Using the Keyboard
10. Check the children
11. Fetch content of a control
12. Window location
13. Move the mouse
14. Move the cursor around the edges
15. Close the window by a mouse click
16. Calculate using the mouse
17. The full calc.pl example
18. Installing Win32::GuiTest
19. Tools
Appendixes
1. Test related CPAN Modules
2. Generic modules
3. Web testing
4. GUI testing
5. Other sources of information
6. Test Realistically
7. Test Unrealistically
8. File System testing