TOC: Python Programming

• Files
  1. File types: Text vs Binary
  2. Open vs. Read vs. Load
  3. Binary files: Images
  4. Reading an Excel file
  5. Reading a YAML file
  6. Read and analyze a text file
  7. Open and read file (easy but not recommended)
  8. Open and read file using with (recommended)
  9. Read file remove newlines
  10. Filename on the command line
  11. Filehandle with return
  12. Read all the lines into a list
  13. Read all the characters into a string (slurp)
  14. Not existing file
  15. Open file exception handling
  16. Open many files - exception handling
  17. Writing to file
  18. Print to file
  19. Append to file
  20. Binary mode
  21. Does file exist? Is it a file?
  22. Direct access of a line in a file
  23. Exercise: count digits
  24. Exercise: remove newlines
  25. Exercise: print lines with Report:
  26. Exercise: color selector
  27. Exercise: ROT13
  28. Exercise: Combine lists
  29. Exercise: Number guessing game - save to file
  30. Solution: count numbers
  31. Solution: remove newlines
  32. Solution: print lines with Report:
  33. Solution: color selector
  34. Solution: ROT13
  35. Solution: Combine lists
  36. Solution: Combine lists with tuple
  37. Filehandle using with and not using it
• Dictionary (hash)
  1. What is a dictionary
  2. When to use dictionaries
  3. Various dictionary examples
  4. Dictionary
  5. Create dictionary
  6. keys
  7. Loop over keys
  8. Loop over dictionary keys
  9. Loop using items
  10. values
  11. Not existing key
  12. Get key
  13. Does the key exist?
  14. Does the value exist?
  15. Delete key
  16. List of dictionaries
  17. Shared dictionary
  18. immutable collection: tuple as dictionary key
  19. immutable numbers: numbers as dictionary key
  20. Sort a dictionary
  21. Sort dictionary values
  22. Sort dictionary by value
  23. Sort dictionary keys by value (another example)
  24. Insertion Order is kept
  25. Change order of keys in dictionary - OrderedDict
  26. Set order of keys in dictionary - OrderedDict
  27. Setdefault
  28. Exercise: count characters
  29. Exercise: count words
  30. Exercise: count words from a file
  31. Exercise: Apache log
  32. Exercise: Combine lists again
  33. Exercise: counting DNA bases
  34. Exercise: Count Amino Acids
  35. Exercise: List of dictionaries
  36. Exercise: Dictionary of dictionaries
  37. Exercise: Age limit with dictionaries
  38. Exercise: Merge files with timestamps
  39. Solution: count characters
  40. Default Dict
  41. Solution: count characters with default dict
  42. Solution: count words (plain)
  43. Solution: count words (defaultdict)
  44. Solution: count words (Counter)
  45. Solution: count words in file
  46. Solution: Apache log
  47. Solution: Apache log using split
  48. Solution: Combine files
  49. Solution: Combine files-improved
  50. Solution: counting DNA bases
  51. Solution: Count Amino Acids
  52. Solution: List of dictionaries
  53. Solution: Dictionary of dictionaries
  54. Solution: Age limit with dictionaries
  55. Solution: Merge files with timestamps
  56. Do not change dictionary in loop
  57. Named tuple (sort of immutable dictionary)
  58. Create dictionary from List
  59. Sort Hungarian letters (lookup table)
• Sets
  1. sets
  2. set operations
  3. Creating a set
  4. Creating a set from a list
  5. Converting set to list
  6. Creating an empty set
  7. Adding an element to a set (add)
  8. Merging one set into another set (update)
  9. set intersection
  10. set subset
  11. set symmetric difference
  12. set union
  13. set relative complement (difference)
  14. Set of numbers
  15. Set of lists
  16. Set of tuples
  17. Create set from List
• Code Reuse
  1. Permutations
  2. k-Permutations
  3. Binomial coefficient
  4. Binomial coefficient - factorial function
  5. k-Permutations - factorial function
  6. Permutations - factorial funcion
  7. mymath module
  8. Permutations - module
  9. k-Permutations - module
  10. Binomial coefficient - module
• Functions (subroutines)
  1. Why use functions?
  2. Defining simple function
  3. Passing positional parameters to a function
  4. Function parameters can be named
  5. Mixing positional and named parameters
  6. Mixing positional and named parameters - order
  7. Default values, optional parameters, optional parameters
  8. Default value in first param
  9. Several defaults, using names
  10. Default list
  11. Arbitrary number of arguments `*`
  12. Arbitrary number of arguments passing a lists
  13. Arbitrary number of arguments passing a tuple
  14. Fixed parmeters before the others
  15. Pass arbitrary number of functions
  16. Arbitrary key-value pairs in parameters `**`
  17. Pass a real dictionary
  18. The dictionary contains copy
  19. The dictionary contains copy but NOT deep copy!
  20. Extra key-value pairs in parameters
  21. Extra key-value pairs in parameters for email
  22. Every parameter option
  23. Duplicate declaration of functions (multiple signatures)
  24. Pylint duplicate declaration
  25. Return more than one value
  26. Recursive factorial
  27. Recursive Fibonacci
  28. Non-recursive Fibonacci
  29. Unbound recursion
  30. Set recurions limit
  31. Variable assignment and change - Immutable
  32. Variable assignment and change - Mutable list
  33. Variable assignment and change - Mutabled dict
  34. Parameter passing of functions
  35. Passing references
  36. Function documentation
  37. Sum ARGV
  38. Copy-paste code
  39. Copy-paste code fixed
  40. Copy-paste code further improvement
  41. Palindrome
  42. Exit vs return vs break and continue
  43. Exercise: statistics
  44. Exercise: Pascal's triangle
  45. Exercise: Pascal's triangle functions
  46. Exercise: recursive dependency tree
  47. Exercise: dependency tree
  48. Exercise: Tower of Hanoi
  49. Exercise: Merge and Bubble sort
  50. Exercise: Refactor previous solutions to use functions
  51. Exercise: Number guessing - functions
  52. Solution: statistics
  53. Solution: Pascal triangle
  54. Solution: Pascal triangle functions
  55. Solution: recursive
  56. Solution: Tower of Hanoi
  57. Solution: Merge and Bubble sort
• Modules
  1. Goal of having modules
  2. Before modules
  3. Create modules
  4. path to load modules from - The module search path
  5. sys.path - the module search path
  6. Project directory layouts
  7. Flat project directory structure
  8. Absolute path
  9. Relative path
  10. Relative path explained
  11. Submodules
  12. Python modules are compiled
  13. How "import" and "from" work?
  14. Execute at import time
  15. Runtime loading of modules
  16. Conditional loading of modules
  17. What is in our namespace?
  18. Runtime import
  19. Duplicate importing of functions
  20. Duplicate importing of functions - solved
  21. Script or library
  22. Script or library - import
  23. Script or library - from import
  24. Scope of import
  25. Import multiple times
  26. Do not import *
  27. Exercise: Number guessing
  28. Exercies: Scripts and modules
  29. Exercise: Module my_sum
  30. Exercise: Convert your script to module
  31. Exercise: Add doctests to your own code
  32. Solution: Module my_sum
  33. Loaded modules and their path
  34. Built-in modules
• PyCharm
  1. PyCharm Intro
  2. PyCharm configure interpreter
  3. PyCharm install modules
  4. PyCharm Project
  5. PyCharm Files
  6. PyCharm - run code
  7. PyCharm - debugging code
  8. PyCharm Terminal
  9. PyCharm Python console at the bottom left
  10. Refactoring example with PyCharm
• Visual Studio Code
  1. VS Code Intro
  2. VS Code Project or Single file
  3. Install the Python Extension
  4. VS Code examples
• Python standard modules (standard packages)
  1. Some Standard packages
• math
  1. math examples
• sys
  1. sys module
  2. Writing to standard error (stderr)
  3. exit prints to STDERR
• os
  1. python which OS are we running on (os, platform)
  2. Current directory (getcwd, pwd, chdir)
  3. OS path
  4. os.path.join
  5. Directory listing
  6. Directory listing using glob
  7. Traverse directory tree - list directories recursively
  8. OS dir (mkdir, makedirs, remove, rmdir)
  9. expanduser - handle tilde ~ the home directory of the user
  10. Get process ID
  11. External command with system
  12. Accessing the system environment variables from Python
  13. Set environment variables on the fly
  14. Reading the .env environment file in Python
  15. Set env and run command
• Pathlib
  1. Pathlib example
  2. Pathlib cwd
  3. Pathlib get extension (suffix)
  4. Pathlib current file
  5. Pathlib parents (dirname)
  6. Pathlib parts (basename)
  7. Pathlib exists
  8. Pathlib iterdir (flat)
  9. Pathlib mkdir (makedir)
• shutil
  1. shutil module
• time
  1. time module
  2. sleep in Python
  3. timer
• datetime
  1. Current date and time datetime now
  2. Converting string to datetime (parsing date and time string)
  3. Parse datetime string with and without timezone information
  4. datetime fromisoformat
  5. date fromisoformat (only date, no time)
  6. datetime arithmeticis (subtract)
  7. Timezone aware datetime
  8. datetime arithmeticis (compare, sort)
  9. datetime arithmeticis (addition)
  10. Rounding datetime object to nearest second (removing microseconds)
  11. Rounding datetime object to date (removing hours, minutes, seconds)
  12. Convert datetime object to date object
  13. Convert datetime object to time object
  14. Today (date)
• subprocess
  1. External CLI tool to demo subprocess
  2. Run with os.system
  3. Run external process let STDOUT and STDERR through
  4. Run external process and capture STDOUT and STDERR separately
  5. Run external process and capture STDOUT and STDERR merged together
  6. subprocess in the background
  7. subprocess collect output while external program is running
  8. Exercise: Processes
  9. Subprocess TBD
• Command line arguments with argparse
  1. Command line arguments
  2. Modules to handle the command line
  3. argparse
  4. Basic usage of argparse
  5. Positional argument
  6. Many positional argument
  7. Convert to integers
  8. Convert to integer
  9. Named arguments
  10. Boolean Flags
  11. Short names
  12. argparse print help explicitely
  13. Argparse xor - mutual exlucise - only one - exactly one
  14. Argparse argument with default and optional value
  15. Conditional required parameter with argparse
  16. Exercise: Command line parameters
  17. Exercise: argparse positional and named
  18. Other
• JSON
  1. JSON - JavaScript Object Notation
  2. JSON dumps
  3. JSON loads
  4. dump
  5. load
  6. Round trip
  7. Pretty print JSON
  8. Serialize Datetime objects in JSON
  9. Sort keys in JSON
  10. Set order of keys in JSON - OrderedDict
  11. Exercise: Counter in JSON
  12. Exercise: Phone book in JSON
  13. Solution: Counter in JSON
  14. Solution: Phone book
• YAML
  1. YAML - YAML Ain't Markup Language
  2. Read YAML
  3. Write YAML
  4. Exercise: Counter in YAML
  5. Exercise: Phone book in YAML
  6. Solution: Counter in YAML
• Exception handling
  1. Hierarchy of calls
  2. Handling errors as return values
  3. Handling errors as exceptions
  4. A simple exception
  5. Prevention
  6. Working on a list
  7. Catch ZeroDivisionError exception
  8. Module to open files and calculate something
  9. File for exception handling example
  10. Open files - exception
  11. Handle divide by zero exception
  12. Handle files - exception
  13. Catch all the exceptions and show their type
  14. List exception types
  15. Hierarchy of Exceptions
  16. Order of exception handling - bad
  17. Order of exception handling - good
  18. How to raise an exception
  19. Raise ValueError exception
  20. Stack trace of exceptions
  21. No need for exception to print Stack trace
  22. Raise Exception from
  23. Exercise: Exception int conversion
  24. Exercise: Raise Exception
  25. Solution: Exception int conversion (specific)
  26. Solution: Exception int conversion (all other)
  27. Solution: Raise Exception
• PyPi - Python Package Index
  1. What is PyPi?
  2. pip
  3. Configure pip on Windows to avoid SSL issues
  4. Upgrade pip
  5. Upgrade PIP on Windows
  6. PYTHONPATH
  7. Requirements
  8. Virtualenv
• Database
  1. Relational Databases (SQL)
  2. NoSQL
  3. Types of NoSQL databases
• SQLite Database Access
  1. SQLite
  2. Connecting to SQLite database
  3. Connecting to in-memory SQLite database
  4. Create TABLE in SQLite
  5. INSERT data into SQLite database
  6. SELECT data from SQLite database
  7. SELECT aggregate data from SQLite database
  8. SELECT data from SQLite database into dictionaries
  9. UPDATE data in SQLite database
  10. A counter
  11. SQLite in-memory AUTOINCREMENT
  12. SQLite in-memory field with DEFAULT value
  13. SQLite transactions
• MySQL
  1. Install MySQL support
  2. Create database user (manually)
  3. Create database (manually)
  4. Create table (manually)
  5. Connect to MySQL
  6. Connect to MySQL and Handle exception
  7. Select data
  8. Select more data
  9. Select all data fetchall
  10. Select some data fetchmany
  11. Select some data WHERE clause
  12. Select into dictionaries
  13. Insert data
  14. Update data
  15. Delete data
  16. Exercise MySQL
  17. Exercise: MySQL Connection
  18. Solution: MySQL Connection
• PostgreSQL
  1. PostgreSQL install
  2. PostgreSQL with Docker compose
  3. Python and Postgresql
  4. PostgreSQL connect
  5. PostgreSQL create table
  6. INSERT
  7. INSERT (from command line)
  8. SELECT
  9. DELETE
• SQLAlchemy
  1. SQLAlchemy hierarchy
  2. SQLAlchemy engine
  3. SQLAlchemy autocommit
  4. SQLAlchemy engine CREATE TABLE
  5. SQLAlchemy engine INSERT
  6. SQLAlchemy engine SELECT
  7. SQLAlchemy engine SELECT all
  8. SQLAlchemy engine SELECT fetchall
  9. SQLAlchemy engine SELECT aggregate
  10. SQLAlchemy engine SELECT IN
  11. SQLAlchemy engine SELECT IN with placeholders
  12. SQLAlchemy engine connection
  13. SQLAlchemy engine transaction
  14. SQLAlchemy engine using context managers
  15. Exercise: Create table
  16. SQLAlchemy Metada
  17. SQLAlchemy types
  18. SQLAlchemy ORM - Object Relational Mapping
  19. SQLAlchemy ORM create
  20. SQLAlchemy ORM schema
  21. SQLAlchemy ORM reflection
  22. SQLAlchemy ORM INSERT after automap
  23. SQLAlchemy ORM INSERT
  24. SQLAlchemy ORM SELECT
  25. SQLAlchemy ORM SELECT cross tables
  26. SQLAlchemy ORM SELECT and INSERT
  27. SQLAlchemy ORM UPDATE
  28. SQLAlchemy ORM logging
  29. Solution: Create table
  30. Exercise: Inspector
  31. SQLAlchemy CREATE and DROP
  32. SQLAlchemy Notes
  33. SQLAlchemy Meta SQLite CREATE
  34. SQLAlchemy Meta Reflection
  35. SQLAlchemy Meta INSERT
  36. SQLAlchemy Meta SELECT
• MongoDB
  1. MongoDB CRUD
  2. Install MongoDB support
  3. MongoDB in Docker compose
  4. Python MongoDB drop database
  5. Python MongoDB insert
  6. MongoDB CLI
  7. Python MongoDB find
  8. Python MongoDB find refine
  9. Python MongoDB update
  10. Python MongoDB remove (delete)
  11. Python MongoDB replace
  12. Python MongoDB upsert
  13. Python Mongodb: TypeError: upsert must be True or False
  14. Python MongoDB Async with motor - connect
  15. Python MongoDB Async with motor - insert and find
• Redis
  1. Redis CLI
  2. Redis list keys
  3. Redis set get
  4. Redis incr
  5. Redis incrby
  6. Redis setex
• Web client - web scraping
  1. get HTML page using urllib
  2. Download image using urllib
  3. get HTML page using requests
  4. Download image using requests
  5. Download image as a stream using requests
  6. Download zip file using requests
  7. Extract zip file
  8. Beautiful Soup to parse HTML
  9. requests - JSON - API
  10. httpbin.org
  11. requests get from httpbin - JSON
  12. requests get IP from httpbin - JSON
  13. requests get JSON User-Agent
  14. requests change User-Agent
  15. requests get header
  16. requests change header
  17. requests post
  18. Interactive Requests
  19. Download the weather - scraping
  20. Download the weather - API call with requests
  21. Download the weather - API call with requests
  22. Tweet
  23. bit.ly
  24. API config file
  25. Exercise: Combine web server and client
• Python Web server
  1. Hello world web
  2. Dump web environment info
  3. Web echo
  4. Web form
  5. Resources
• Networking
  1. Secure shell
  2. ssh
  3. ssh from Windows
  4. Parallel ssh
  5. telnet
  6. prompt for password
  7. ftp
• Interactive shell
  1. The Python interactive shell
  2. REPL - Read Evaluate Print Loop
  3. Using Modules
  4. Getting help
  5. Exercise: Interactive shell
• Testing Demo
  1. Testing Flask
  2. How do you test your code?
  3. What is testing?
  4. What is testing really?
  5. Testing demo tools
  6. Testing demo methodology
  7. Testing demo - AUT - Application Under Test
  8. Testing demo - use the module
  9. Testing demo: doctest
  10. Testing demo: doctest with failure
  11. Testing demo: Unittest success
  12. Testing demo: Unittest failure
  13. Testing demo: pytest using classes
  14. Testing demo: pytest using classes - failure
  15. Testing demo: pytest without classes
  16. Testing demo: pytest without classes failure
  17. Testing demo: Failure in one sub
  18. Testing demo: pytest run doctests
  19. Testing demo: pytest run unittest
  20. Test demo: test coverage
  21. Exercise: Testing demo - anagrams
  22. Exercise: Test previous solutions
  23. Solution: Testing demo
• GitHub
  1. GitHub data
  2. GitHub API: REST vs GraphQL
  3. Where is it used
  4. GitHub get organization members
  5. Details about an orgarnization REST
  6. Details about an user REST
  7. REST - List of repositories by organization (pagination!)
  8. REST - List of reposistories by user (pagination!)
  9. GraphQL - List repositories by organization
  10. GitHub API KEY (PERSONAL TOKEN)
  11. GitHub REST API
  12. GitHub REST API execute query
  13. GitHub API GraphQL
  14. GitHub GraphQL explorer
  15. GitHub GraphQL execute query
  16. GitHub GraphQL execute query async
  17. GitHub GraphQL who am i
  18. GitHub GraphQL list my repositories
  19. GitHub GraphQL list of repositories by username
  20. GitHub GraphQL list issues by username
  21. GitHub GraphQL list issues using parameter
  22. GitHub GraphQL list issues using several parameters
  23. GitHub GraphQL contribution counts
  24. GitHub GraphQL list Pull-Requests
  25. GitHub GraphSQL paging using cursor
  26. GitHub GraphQL activities
• Types in Python
  1. mypy
  2. Changing types
  3. Changing types when reading a number
  4. Types of variables
  5. Types of function parameters
  6. Types function returns None or bool
  7. Types used properly
  8. TODO: mypy
• Testing Intro
  1. The software testing equasion
  2. The software testing equasion (fixed)
  3. The pieces of your software?
  4. Manual testing
  5. What to tests?
  6. Continuous Integration
• Logging
  1. Simple logging
  2. Simple logging - set level
  3. Simple logging to a file
  4. Simple logging format
  5. Simple logging change date format
  6. getLogger
  7. Time-based logrotation
  8. Size-based logrotation
• Closures
  1. Counter local - not working
  2. Counter with global
  3. Create incrementors
  4. Create internal function
  5. Create function by a function
  6. Create function with parameters
  7. Counter closure
  8. Make incrementor with def (closure)
  9. Make incrementor with lambda
  10. Exercise: closure bank
  11. Exercise: counter with parameter
  12. Solution: closure bank
  13. Solution: counter with parameter
• Decorators
  1. Decorators: simple example
  2. Decorators - Flask
  3. Decorators - Pytest
  4. Decorators caching - no cache
  5. Decorators caching - with cache
  6. LRU - Least recently used cache
  7. LRU - Least recently used cache
  8. OOP - classmethod - staticmethod
  9. Use cases for decorators in Python
  10. Function assignment
  11. Function assignment - alias print to say
  12. Function assignment - don't do this
  13. Passing functions as parameters
  14. Traversing directory tree
  15. Declaring Functions inside other function
  16. Returning a new function from a function
  17. Returning a closure
  18. Decorator
  19. Decorator Demo
  20. Decorator to register function
  21. A recursive Fibonacci
  22. trace fibo
  23. tron decorator
  24. Decorate with direct call
  25. Decorate with parameter
  26. Decorator accepting parameter
  27. Decorate function with any signature
  28. Decorate function with any signature - implementation
  29. Decorate function with any signature - skeleton
  30. Decorate function with any signature - skeleton with name
  31. Functool - partial
  32. Exercise: Logger decorator
  33. Exercise: decorators decorator
  34. Solution: Logger decorator
  35. Solution: Logger decorator (testing)
  36. Solution decorators decorator
  37. A list of functions
• Context managers (with statement)
  1. Why use context managers?
  2. Using Context Manager
  3. Context Manager examples
  4. cd in a function
  5. open in function
  6. open in for loop
  7. open in function using with
  8. Plain context manager
  9. Param context manager
  10. Context manager that returns a value
  11. Use my tempdir - return
  12. Use my tempdir - exception
  13. cwd context manager
  14. tempdir context manager
  15. Context manager with class
  16. Context managers with class
  17. Context manager: with for file
  18. With - context managers
  19. Exercise: Context manager
  20. Exercise: Tempdir on Windows
  21. Solution: Context manager
• Advanced lists
  1. Change list while looping: endless list
  2. Change list while looping
  3. Copy list before iteration
  4. for with flag
  5. for else
  6. enumerate
  7. do while
  8. list slice is copy
• Advanced Exception handling
  1. Exceptions else
  2. Exceptions finally
  3. Exit and finally
  4. Catching exceptions
  5. Home made exception
  6. Home made exception with attributes
  7. Home made exception hierarcy
  8. Home made exception hierarcy - 1
  9. Home made exception hierarcy - 2
  10. Home made exception hierarcy - 3
  11. Exercise: spacefight with exceptions
  12. Exercies: Raise My Exception
  13. Solution: spacefight with exceptions
  14. Solution: Raise My Exception
  15. Exception finally return
• Warnings
  1. Warnings
• CSV
  1. What is a CSV file?
  2. CSV file without title row
  3. CSV file with header
  4. Read CSV file into lists
  5. CSV with newlines missing closing quote
  6. CSV to dictionary
  7. CSV Attributes
  8. CSV dialects
  9. Reading CSV the naive way
  10. CSV with quotes and newlines
  11. Reading a CSV file
  12. Exercise: CSV as dictionary of dictionaries
  13. Exercise: CSV as dictionary of tuples of dictionaries
  14. Exercise: count row length in csv files
  15. Solution: CSV as dictionary of dictionaries
  16. Solution: CSV as dictionary of tuples of dictionaries
  17. Solution: count row length in csv files
• Excel
  1. Spreadsheets
  2. Python Excel
  3. Create an Excel file from scratch
  4. Worksheets in Excel
  5. Add expressions to Excel
  6. Format field
  7. Number series and chart
  8. Read Excel file
  9. Update Excel file
  10. Barchart
  11. Exercise: Excel
• XML
  1. XML Data
  2. Expat - Callbacks
  3. XML DOM - Document Object Model
  4. XML SAX - Simple API for XML
  5. SAX collect
  6. XML elementtree
• Matplotlib
  1. About Matplotlib
  2. Matplotlib Line
  3. Matplotlib Line with dates
  4. Matplotlib Simple Pie
  5. Matplotlib Simple Pie with params
  6. Matplotlib Pie
  7. Matplotlib Pie (test cases)
  8. Plot, scatter, histogram
• Seaborn
  1. Searborn use examples
  2. Seaborn tip
  3. Seaborn Anscombes Quartet
• Jupyter notebooks
  1. What is Jupyter Notebook?
  2. Jupyter on Windows
  3. Jupyter on Linux and OSX
  4. Jupyter New notebook
  5. Jupyter Notebook file format
  6. Jupyter notebook edit and command mode
  7. Jupyter notebook Intellisense (TAB completition)
  8. Jupyter add
  9. Planets
  10. Jupyter input
  11. File chooser
  12. IPy Widgets
  13. Jupyter Notebook and Markdown
  14. Latex in Jupyter Notebook
  15. Jupyter StackOverflow
  16. Use SciPy image
  17. Create Random image
  18. Load image using OpenCV
  19. Genes using Jupyter
  20. More Jupyter examples
  21. Jupyter Notebook: run on other port
  22. Jupyter Notebook: public IP
  23. Other Jupyter Notebook Kernels
  24. Jupyter Notebook convert to other format (nbconvert)
  25. Jupyter Notebook extensions
  26. Jupyter notebook Checkpoints
  27. Jupyter notebook autoreload
• Tox
  1. Tox Examples
• Selenium
  1. Selenium installation
  2. Get started with Selenium
  3. Selenium Headless Screenshot
• Playwright
  1. Playwright installation
  2. Playwright demo
  3. Playwright screenshot
• Advancted functions
  1. Variable scopes
  2. Name resolution order (LEGB)
  3. Scoping: global seen from fuction
  4. Assignment creates local scope
  5. Local scope gone wrong
  6. Changing global variable from a function
  7. Global variables mutable in functions
  8. Scoping issues
  9. sub in sub
  10. Scoping sub in sub (enclosing scope)
  11. Function objects
  12. Functions are created at run time
  13. Mutable default
  14. Use None as default parameter
  15. Inner function created every time the outer function runs
  16. Static variable
  17. Static variable in generated function
  18. Inspect
• Variable number of function arguments
  1. Python function arguments - a reminder
  2. Functions with unknown number of argumerns
  3. Variable length argument list with * and **
  4. Passing arguments as they were received (but incorrectly)
  5. Unpacking args before passing them on
  6. Exercise: implement the my_sum function
  7. Solution: implement the my_sum function
  8. Exercise: implement the reduce function
  9. Soluton: implement the reduce function
  10. Exercise: sort pairs
  11. Solution: sort pairs
• Python Packages
  1. Why Create package
  2. Create package
  3. Internal usage
  4. use module in package - relative path
  5. use package (does not work)
  6. package importing (and exporting) module
  7. use package (module) with import
  8. use package with import
  9. Creating an installable Python package
  10. Create tar.gz file
  11. Install Package
  12. Dependencies
  13. Add README file
  14. Add README file (setup.py)
  15. Include executables
  16. Add tests
  17. Add tests calc
  18. Add tests all
  19. setup.py
  20. Run tests and create package
  21. Exercise: package
  22. Exercise: create executable
• Distribution of Python code
  1. Distribution demo 1
  2. Distribution demo 2
  3. Distribution demo 3
• Distribute Python application as an exe
  1. Packaging applications (creating executable binaries)
  2. Using PyInstaller
  3. Other PyInstaller examples
  4. Other
  5. Py2app for Mac
• Ctypes
  1. ctypes - hello
  2. concat
  3. links
• 2to3
  1. Convertig from Python 2 to Python 3
  2. division
  3. print in Python 2
  4. print in Python 3
  5. input and raw_input
  6. Code that works on both 2 and 3
  7. Compare different types
  8. Octal numbers
  9. 2to3 Resources
• Design Patterns
  1. What are Design Patterns?
  2. Don't replace built-in objects
  3. Facade - simple interface to complex system
  4. Monkey Patching
  5. Creation DPs "Just One"
  6. Singleton
  7. Monostate (Borg)
  8. Dispatch table
• Parallel
  1. Types of Problems
  2. Types of solutions
  3. Time slicing
  4. Concurrencies
  5. How many parallels to use?
  6. Dividing jobs
  7. Performance Monitoring
• Threads
  1. Python Threading docs
  2. Threaded counters
  3. Simple threaded counters
  4. Simple threaded counters (parameterized)
  5. Pass parameters to threads - Counter with attributes
  6. Create a central counter
  7. Lock - acquire - release
  8. Counter - plain
  9. GIL - Global Interpreter Lock
  10. Thread load
  11. Exercise: thread files
  12. Exercise: thread URL requests.
  13. Exercise: thread queue
  14. Solution: thread queue
  15. Solution: thread URL requests.
  16. Concurrency
• Forking
  1. Fork
  2. Forking
  3. Fork skeleton
  4. Fork with load
  5. Fork load results
  6. Marshalling / Serialization
  7. Fork with random
  8. Exercise: fork return data
  9. Solution: fork return data
• Asynchronous programming with AsyncIO
  1. Use cases for Async
  2. Event loop
  3. Print sync
  4. Print async
  5. Sync sleep
  6. Async sleep
  7. Sync sleep in loop
  8. Async sleep in loop
  9. Async sleep in loop with gather
  10. coroutines
  11. Async Tasks
  12. Count Async
  13. Passing the baton while sleeping 0 sec
  14. Async sleep in a queue
  15. Async http
  16. Sync http requests
  17. Async http requests
  18. Async http requests with queue
  19. Sync chores
  20. Async chores
• Async more
  1. Explanation
  2. Coroutines
  3. More about asyncio
  4. unsync
  5. Async files
  6. Async example
• Asynchronus programming with Twisted
  1. About Twisted
  2. Echo
  3. Echo with log
  4. Simple web client
  5. Web client
• Multiprocess
  1. Multiprocess CPU count
  2. Multiprocess N files: Pool
  3. Multiprocess load
  4. Multiprocess: Pool
  5. Multiprocess load async
  6. Multiprocess and logging
  7. Exercise: Process N files in parallel
  8. Exercise: Process N Excel files in parallel
  9. Exercise: Fetch URLs in parallel
  10. Exercise: Fetch URLs from one site.
  11. Solution: Process N files in parallel
  12. Solution: Fetch URLs in parallel
• Multitasking
  1. What is Multitasking?
  2. Multitasking example
  3. Multitasking example with wait
  4. Multitaksing - second loop waits for first one
  5. Multitasking counter
  6. Multitasking counter with thread locking
• Improving Performance - Optimizing code
  1. Problems
  2. Optimization strategy
  3. Locate the source of the problem
  4. Optimizing tactics
  5. DSU: Decorate Sort Undecorate
  6. Profile code
  7. Slow example
  8. profile slow code
  9. cProfile slow code
  10. Benchmarking
  11. Benchmarking subs
  12. Counting words - which implementation is faster?
  13. for loop or reduce to add numbers?
  14. Levenshtein distance
  15. Generate words
  16. Levenshtein - pylev
  17. Levenshtein - editdistance
  18. Editdistance benchmark
  19. A Tool to Generate text files
  20. Count characters
  21. Memory leak
  22. Garbage collection
  23. Weak reference
  24. Exercise: benchmark list-comprehension, map, for
  25. Exercise: Benchmark Levenshtein
  26. Exercise: sort files
  27. Exercise: compare split words:
  28. Exercise: count words
• GUI with Python/Tk
  1. Sample Tk Demo app
  2. Simple file dialog
  3. GUI Toolkits
  4. Installation
  5. Python Tk Documentation
  6. Python Tk Button
  7. Python Tk Button with action
  8. Python Tk Label
  9. Python Tk Label - font size and color
  10. Python Tk echo - change text of label
  11. Python Tk Keybinding
  12. Python Tk Mouse clicks
  13. Python Tk Mouse movements (motions)
  14. Python Tk Entry (one-line text entry)
  15. Python Tk Entry for passwords and other secrets (hidden text)
  16. Python Tk Checkbox
  17. Python Tk Radiobutton
  18. Python Tk Listbox
  19. Python Tk Listbox Multiple
  20. Python Tk Menubar
  21. Python Tk Text
  22. Python Tk Dialogs
  23. Python Tk simple dialog to get a single string, int, or float
  24. Python Tk Filedialog
  25. Python Tk messagebox
  26. Python Tk - custom simple dialog with its own widgets and buttons
  27. Python Tk Combobox
  28. Python Tk OptionMenu
  29. Python Tk Scale
  30. Python Tk Progressbar
  31. Python Tk Frame
  32. Python Tk display images using Canvas
  33. Python Tk display Hebrew text (right to left)
  34. Python Tk Colorchooser
  35. Python Tk Timer event (after)
  36. Python Tk Class-based Label + Button
  37. Tk: Runner
  38. Tk: Runner with threads
  39. Tk: Old Simple Tk app with class
  40. Tk: Old Hello World
  41. Tk: Old Quit button
  42. Tk: Old File selector
  43. Tk: Old Checkbox
  44. Tk: Old Getting started with Tk
  45. Exercise: Tk - Calculator one line
  46. Exercise: Tk - Calculator with buttons
  47. Exercise: Tk - Convert between CSV and Excel files
  48. Exercise: Tk - Shopping list
  49. Exercise: Tk - TODO list
  50. Exercise: Tk - Notepad
  51. Exercise: Tk - Copy files
  52. Exercise: Tk - Implement Master Mind board
  53. Exercise: Tk - a GUI for a grep-like application
  54. Solution: Tk - Calculator one line
  55. Solution: Tk - Calculator with buttons
  56. Solution: Tk - Convert between CSV and Excel files
  57. Solution: Tk - Implement Master Mind board
  58. Solution: Tk - Notepad
• Python Pitfalls
  1. Reuse of existing module name
  2. Use the same name more than once
  3. Compare string and number
  4. Compare different types
  5. Sort mixed data
• Linters
  1. Static Code Analyzis - Linters
  2. PEP8
  3. F811 - redefinition of unused
  4. Warn when Redefining functions
• Python .NET
  1. IronPython
  2. Use .NET libraries from Python
  3. Python and .NET console
  4. Python and .NET examples
  5. Exercise Python and .NET
• Python and Java
  1. Jython
  2. Calling Java from Python
• Jython - Python running on the JVM
  1. Jython Installation
  2. Jython Installation
  3. Jython load Java class
  4. Jython load Java class in code
  5. Jython test Java class
• Signals
  1. Signals and Python
  2. Sending Signal
  3. Catching Signal
  4. Catching Ctrl-C on Unix
  5. Catching Ctrl-C on Unix confirm
  6. Alarm signal and timeouts
  7. Exercise: Catching Ctrl-C on Unix 2nd time
  8. Exercise: Signals
  9. Ctrl-z
• Data Science
  1. Data Scince Resources
• PIL - Pillow
  1. Install Pillow
  2. Create First Image
  3. Write Text on Image
  4. Select font for Text on Image
  5. Font directories
  6. Get size of an Image
  7. Get size of text
  8. Resize an existing Image
  9. Crop an existing Image
  10. Combine two images
  11. Rotated text
  12. Rotated text in top-right corner
  13. Embed image (put one image on another one)
  14. Draw a triangle
  15. Draw a triangle and write text in it
  16. Draw a triangle and write rotated text in it
  17. Draw a rectangle
  18. Draw circle
  19. Draw heart
  20. Rectangle with rounded corners
  21. TODO
• FAQ
  1. How not to name example scirpts?
  2. Platform independent code
  3. How to profile a python code to find causes of slowness?
  4. pdb = Python Debugger
  5. Avoid Redefining functions
• Appendix
  1. print_function
  2. Dividers (no break or continue)
  3. Lambdas
  4. Abstract Class
  5. Remove file
  6. Modules: more
  7. import hooks
  8. Python resources
  9. Progress bar
  10. from __future__
  11. Variable scope
  12. scope
  13. type
  14. Look deeper in a list
  15. Exercise: iterators - count
• Algorithm
  1. Exercise: Find the odd value
  2. Solution: Find the odd value
  3. Exercise: Generalized find the odd value
  4. Solution: Generlized Find the odd value
  5. Exercise: Shortest sublist with sum over limit
  6. Solution: Shortest sublist with sum over limit
• Scapy
  1. Virtualenv for root
  2. Install Scapy
  3. Tcpdump
  4. Scapy ping an IP using ICMP
  5. Scapy ping ICMP
  6. Scapy display (show)
  7. Scapy ip
  8. Scapy Configuration
  9. Scapy List interfaces
  10. Scapy ping-pong
  11. Scapy sniffing - capturing packets
  12. Scapy ping between two IPs that do not belong to my server
  13. Scapy Traceroute
  14. Scapy TCP port 80
  15. Scapy Layers
  16. Scapy HTTP
• Turtle
  1. Line
  2. Square (forward, left)
  3. Circle
  4. Square using the circle command
  5. Triangle
  6. Move turtle without drawing (penup, pendown, goto)
  7. Turtle house
  8. Turtle screen
  9. Tutle onclick - mouse click event
  10. Turle Commands
  11. Turtle Colors
  12. Turtle shape
  13. Turtle speed
  14. Undo the turtle
  15. Create new turtle
  16. Clone a turtle
• Refactor
  1. Refactoring example - change variable name
  2. How to Refactor
  3. Exercise: Fix deep indentation
• Machine Learning
  1. Types of Machine Learning algorithms
  2. ML - Supervised Learning
  3. Regression problem
  4. Classification problem
  5. ML - Unsupervised Learning
  6. Linear regression with sklearn
  7. Split data set
  8. Food-truck linear regression
  9. Basic Classification example
  10. Kaggle
  11. Kaggle - USA housing listing
  12. Kaggle - Iris
• Machine Learning 2
  1. Number of features
  2. Linear regression
  3. Cost function
  4. Gradient descent
  5. Matrices
  6. Machine Learning - Multiple features
  7. Feature Scaling
  8. Gradient Descent - Learning Rate
  9. Features
  10. Polynomial Regression
  11. Normal Equation
  12. Multiple features
  13. Logistic regression (for classification)
  14. Multi-feature Classification (Iris)
  15. Kaggle - Melbourne housing listing
  16. Machine Learning Resources
  17. Regression Analyzis
  18. Classification Analysis
  19. Unbiased evaluation of a model
  20. Splitting data
  21. Model selection and validation
  22. K-fold valiadtion
  23. Learning Curves
  24. Hypermatameter tuning (optimization)
  25. The k-Nearest Neighbors (kNN)
  26. K-Means Clustering
  27. Boston housing prices
  28. Decision Tree
  29. Random Forrest
  30. Resnet 50
• Octave
  1. Octave install
  2. Octave interactive shell
  3. Math
  4. Logical operations
  5. Octave change interactive prompt
  6. Plotting data
  7. Flow Control
• OpenCV
  1. OpenCV Installation
  2. Download images
  3. OpenCV Read image
  4. OpenCV - read image and fail
  5. OpenCV resize (rescale) images
  6. OpenCV cropping image
  7. OpenCV crop, resize and save image
  8. OpenCV - draw on new images
  9. OpenCV - draw rectangle
  10. OpenCV - draw circle
  11. OpenCV convert BGR to greyscale and back to BGR
  12. OpenCV blur image
  13. OpenCV Gaussian blur
  14. OpenCV Median blur
  15. OpenCV Bilateral blur
  16. OpenCV blur part of an image
  17. OpenCV - Thresholding
  18. OpenCV - Simple Thresholding
  19. OpenCV - Adaptive Thresholding
  20. OpenCV - Gradients and Edges
  21. OpenCV - finding edges using Canny
  22. OpenCV - Laplacian Gradients
  23. OpenCV - Sobel Gradients
  24. OpenCV - Canny Dilate Erode
  25. OpenCV Move of an Image
  26. OpenCV - Rotate Image
  27. OpenCV - Flip Image
  28. OpenCV - Contours
  29. OpenCV - Draw contours of an image on a blank image
  30. OpenCV - ColorSpaces
  31. OpenCV - BGR to RGB
  32. OpenCV - BGR to HSV
  33. OpenCV - BGR to LAB
  34. OpenCV - Split and merge color channels
  35. OpenCV - bitwise operations
  36. OpenCV - masking
  37. OpenCV - grayscale historgrams
  38. OpenCV - color historgrams
  39. OpenCV - Face detection
  40. OpenCV - Face recognition
  41. Jupyter notebook
  42. Download movies
  43. OpenCV Read video
  44. Show images - accept key presses
  45. OpenCV Resources
• Overview of Python syntax
  1. Scalars
  2. Numbers
  3. Strings
  4. int - float - string conversion
  5. Booleans
  6. Lists
  7. Queue and Stack
  8. Dictionaries
  9. Tuples
  10. Sets
  11. I/O
  12. STDIN - Standard input
  13. CLI
  14. Control flow
  15. While - Loops
  16. For - Loops
  17. Conditionals
  18. Comparision operators
  19. Boolean operators
  20. The conditional (ternary) operator
  21. Random Values
  22. Math
  23. Exceptions
• Serialization (Marshalling)
  1. Why Serialization is needed?
  2. Questions to ask
  3. Various tools for serialization
  4. Serialization with h5py
  5. Serialization of single Numpy array
  6. Serialization of multiple Numpy arrays
• MLOps
  1. The goal of MLOps
  2. Agile Manifesto
  3. DevOps
  4. Machine Learning Operations
  5. What is MLOps?
  6. MLOps community
  7. Silicon
  8. Off-line (batch) vs On-line (streaming) learning
  9. Security question of MLOps
  10. Data
  11. MLOps progress
  12. Reload modules in Jupyter Notebook
  13. Testing ML
  14. What to track?
  15. What are the inputs and what are the artifacts?
  16. Tooling for MLOps
  17. DVC
  18. Data Pipelines (Workflow management)
  19. MLFlow
  20. MLFLow Tracking server backends
  21. MLFlow Tracking
  22. MLFlow Projects
  23. MLFlow Models
  24. Resources
  25. Goals of SCM
  26. MLOps notes
• Python types at PyWeb 2025.01
  1. Python types at PyWeb 2025.01
  2. The Answer
  3. Add
  4. Add in function
  5. Shift-left (testing, programming)
  6. Function with type annotation
  7. Use mypy
  8. How add type annotation?
  9. Infer (deduct) the type
  10. Type in unannotated function
  11. Built-in types
  12. Complex types
  13. Either this or that type (Union)
  14. Optional type (variable can also be None)
  15. Define type alias
  16. Define complex type alias
  17. Define type for enum and complex dictionary
  18. mypy generics - plain
  19. mypy generics - cannot be any type
  20. mypy generics - limit the types by listing
  21. mypy generics - limit by functionality
  22. Two variables of the same and different types
  23. mypy suggestions
  24. The end
  25. Bloopers
  26. Define the type of variables
  27. Define the return type
  28. Complex types for Python 3.8 and before
  29. Either this or that type for Python before 3.10
• Python types at PyWeb 2025.05
  1. Python types at PyWeb 2025.05
  2. The Answer
  3. Add
  4. Add in function
  5. Shift-left (testing, programming)
  6. Function with type annotation
  7. Use mypy
  8. How add type annotation?
  9. Infer (deduct) the type
  10. Type in unannotated function
  11. Type in annotated function
  12. Built-in types
  13. Complex types
  14. Either this or that type (Union)
  15. Either this or that type for Python before 3.10
  16. Optional type (variable can also be None)
  17. Define type alias
  18. Define complex type alias
  19. Define type for enum and complex dictionary
  20. mypy generics - plain
  21. mypy generics - cannot be any type
  22. mypy generics - limit the types by listing
  23. mypy generics - limit by functionality
  24. Two variables of the same and different types
  25. mypy suggestions
  26. The end
  27. Bloopers
  28. Define the type of variables
  29. Define the return type
  30. Complex types for Python 3.8 and before
• Other slides
  1. Other slides
  2. Atom for Python
  3. IDLE - Integrated DeveLopment Environment
  4. sh-bang - executable on Linux/Apple
  5. pydoc
  6. Spyder Intro
  7. Interactive Debugging
  8. Parameter passing
  9. Command line arguments and main
  10. Name of the current function in Python
  11. Name of the caller function in Python
  12. Stack trace in Python using inspect
  13. SAX with coroutine
  14. Getting the class name of an object
  15. iterator - pairwise
  16. iterator - grouped
  17. itertools - groupby
  18. Circular references
  19. Context managers: with (file) experiments
  20. itertools - izip
  21. mixing iterators
  22. mixing iterators
  23. itertools - pairwise
  24. itertools - grouped
  25. range vs xrange in Python
  26. profile (with hotshot) slow code
  27. Create class with metaclass
  28. Python Descriptors
  29. alter iterator
  30. Create a counter queue
  31. A Queue of tasks
  32. Python from .NET
  33. assert to verify values
  34. mycalc as a self testing module
  35. doctest
  36. Export import
  37. Export import with __all__
  38. import module
  39. deep copy list
  40. deep copy dictionary
  41. Matplotlib subplot
  42. Jupyter StackOverflow - historgram
  43. Jupyter StackOverflow - OpenSourcer
  44. Jupyter StackOverflow - cross tabulation
  45. Jupyter StackOverflow - salaries
  46. Jupyter StackOverflow - replace values
  47. NameError
  48. UnboundLocalError
  49. Insert element in sorted list using bisect
  50. Insert element in sorted list using insort
  51. Classes
  52. Create a class inherit from object
  53. Gravatar in Python
  54. Debug with ptpython
  55. REPL - Interactive debugging with ptpython
  56. Print in color on the CLI
  57. Easy Install
  58. sorting with sorted using a key
  59. get and set locale
  60. Modify time anomality
  61. Some strange code
  62. is vs ==