1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
| import sys
import os
import re
if not[3 python2.7*
lrwxrwxrwx 1 root root 9 xxx xx xxxx python2 -> python2.7*
-rwxr-xr-x 1 root root 3345416 xxx xx xxxx python2.7*
lrwxrwxrwx 1 root root 9 xxx xx xxxx python3 -> python3.5*
-rwxr-xr-x 2 root root 3709944 xxx xx xxxx python3.5*
-rwxr-xr-x 2 root root 3709944 xxx xx xxxx python3.5m*
lrwxrwxrwx 1 root root 10 xxx xx xxxx python3m -> python3.5m*
Windows You could install either: - "Anaconda Distribution" [See previous section]
- Plain Python from Python Software Foundation @ //www.python.org/download/, download the 32-bit or 64-bit MSI installer, and run the downloaded installer.
- Under the Cygwin [Unix environment for Windows] and install Python [under the "devel" category].
Mac OS X[TODO] DocumentationPython documentation and language reference are provided online @ //docs.python.org. Getting Started with Python InterpreterStart the Interactive Python InterpreterYou can run the "Python Interpreter" in
interactive mode under a "Command-Line Shell" [such as Anaconda Prompt, Windows' CMD, Mac OS X's Terminal, Ubuntu's Bash Shell]: $ python
Python 3.7.0
......
Type "help", "copyright", "credits" or "license" for more information.
>>>The Python's command prompt is denoted as >>>. You can enter Python statement at the Python's command prompt, e.g., >>> print['hello, world']
hello, world
>>> print[2 ** 88]
309485009821345068724781056
>>> print[8.01234567890123456789]
8.012345678901234
>>> print[[1+2j] * [3+4j]]
[-5+10j]
>>> x = 123
>>> x
123
>>> msg = 'hi!'
>>> msg
'hi!'
>>> exit[] To exit Python Interpreter: - exit[]
- [Mac OS X and Ubuntu] Ctrl-D
- [Windows] Ctrl-Z followed by Enter
Writing and Running Python ScriptsFirst Python Script -
hello.pyUse a programming text editor to write the following Python script and save as "hello.py" in a directory of your choice:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
| myStr = 'Hello, world'
print[myStr]
myInt = 2 ** 88
print[myInt]
myFloat = 8.01234567890123456789
print[myFloat]
myComplex = [1+2j] / [3-4j]
print[myComplex]
myLst = [11, 22, 33]
print[myLst[1]]
| How it Works- By convention, Python script [module] filenames are in all-lowercase [e.g., hello].
- EOL Comment: Statements beginning with a # until the end-of-line [EOL] are comments.
- #!/usr/bin/env python3 [Line 1] is applicable to the Unix environment only. It is known as the Hash-Bang [or She-Bang] for specifying the location of Python Interpreter, so that the script can be executed
directly as a standalone program.
- # -*- coding: UTF-8 -*- [Line 2, optional] specifies the source encoding scheme for saving the source file. We choose and recommend UTF-8 for internationalization. This special format is recognized by many popular editors for saving the source code in the specified encoding format.
- """ hello ...... """ [Line 3-5]: The script begins by the so-called doc-string to provide the documentation for this Python
module. Doc-string is typically a multi-line string [delimited by triple-single or triple-double quoted], which can be extracted from the source file to create documentation.
- Variables: We create variables myStr, myInt, myFloat, myComplex, myLst [Line 6, 8, 10, 12, 14] by assignment values into them.
- Python's strings can be enclosed with single quotes '...' [Line 6] or double quotes "...".
- Python's integer is unlimited in size [Line 8].
- Python support floating-point numbers [Line 10].
- Python supports complex numbers [Line 12] and other high-level data types.
- Python supports a dynamic array called list [Line 14], represented by lst=[v1, v2, ..., vn]. The element can be retrieved via index lst[i] [Line 15].
- print[aVar]: The print[] function can be used to print the value of a variable to the console.
Expected OutputThe expected outputs are: Hello, world
309485009821345068724781056
8.012345678901234
[-0.2+0.4j]
22Running Python
ScriptsYou can develop/run a Python script in many ways - explained in the following sections. Running Python Scripts in Command-Line Shell [Anaconda Prompt, CMD, Terminal, Bash]You can run a python script via the Python Interpreter under the Command-Line Shell: $ cd
$ python hello.py Unix's Executable Shell ScriptIn Linux/Mac OS X, you can turn a Python script into an executable program [called Shell Script or Executable Script] by: - Start with a line beginning
with #! [called "hash-bang" or "she-bang"], followed by the full-path name to the Python Interpreter, e.g., ...... To locate the Python Interpreter, use command "which python" or "which python3".
- Make the file executable via chmod [change file mode] command: $ cd /path/to/project-directory
$ chmod u+x hello.py
$ ls -l hello.py
-rwxrw-r-- 1 uuuu gggg 314 Nov 4 13:21 hello.py
- You can then run the Python script just like any executable programs. The system will look for the Python Interpreter from the she-bang line. $ cd /path/to/project-directory
$ ./hello.py
The drawback is that you have to hard code the path to
the Python Interpreter, which may prevent the program from being portable across different machines. Alternatively, you can use the following to pick up the Python Interpreter from the environment: ......The env utility will locate the Python Interpreter [from the PATH entries]. This approach is recommended as it does not hard code the Python's path. Windows' Exeutable ProgramIn Windows, you can associate ".py" file extension with the Python
Interpretable, to make the Python script executable. Running Python Scripts inside Python's InterpreterTo run a script "hello.py" inside Python's Interpreter: $ python3
......
>>> exec[open['/path/to/hello.py'].read[]]
$ python2
......
>>> execfile['/path/to/hello.py']
>>> exec[open['/path/to/hello.py']]- You can use either absolute or relative path for the filename. But, '~' [for home directory] does not work?!
- The open[] built-in function opens the file, in default read-only mode; the read[] function reads the entire file.
Interactive Development Environment [IDE]Using
an IDE with graphic debugging can greatly improve on your productivity. For beginners, I recommend: - Python Interpreter [as described above]
- Python IDLE
- Jupyter Notebook [especially for Data Analytics]
For Webapp developers, I recommend: - Eclipse with PyDev
- PyCharm
See "Python IDE and Debuggers" for
details. Python Basic SyntaxesCommentsA Python comment begins with a hash sign [#] and last till the end of the current line. Comments are ignored by the Python Interpreter, but they are critical in providing explanation and documentation for others [and yourself three days later] to read your program. Use comments liberally. There is NO multi-line comment in Python?! [C/C++/Java supports multi-line comments via /* ... */.] Statements
A Python statement is delimited by a newline. A statement cannot cross line boundaries, except: - An expression in parentheses [], square bracket [], and curly braces {} can span multiple lines.
- A backslash [\] at the end of the line denotes continuation to the next line. This is an old rule and is NOT recommended as it is error-prone.
Unlike C/C++/C#/Java, you don't place a semicolon [;] at the end of a Python statement. But you
can place multiple statements on a single line, separated by semicolon [;]. For examples, >>> x = 1
>>> print[x]
1
>>> x + 1
2
>>> y = x / 2
>>> y
0.5
>>> print[x]; print[x+1]; print[x+2]
1
2
3
>>> x = [1,
22,
333]
>>> x
[1, 22, 333]
>>> x = {'name':'Peter',
'gender':'male',
'age':21
}
>>> x
{'name': 'Peter', 'gender': 'male', 'age': 21}
>>> x =[1 +
2
+ 3
-
4]
>>> x
2
>>> s = ['testing '
'hello, '
'world!']
>>> s
'testing hello, world!'Block, Indentation and Compound StatementsA block is a group of statements executing as a unit. Unlike C/C++/C#/Java, which use braces {} to group statements in a body block, Python uses indentation for body block. In other words, indentation is syntactically significant in Python - the body block must be properly indented. This is a good syntax to force you to indent the
blocks correctly for ease of understanding!!! A compound statement, such as conditional [if-else], loop [while, for] and function definition [def], begins with a header line terminated with a colon [:]; followed by the indented body block, as follows: header_1:
statement_1_1
statement_1_2
......
header_2:
statement_2_1
statement_2_2
......
header_1: statement_1_1
header_2: statement_2_1; statement_2_2; ......For examples, x = 0
if x == 0:
print['x is zero']
else:
print['x is not zero']
if x == 0: print['x is zero']
else: print['x is not zero']
sum = 0
number = 1
while number sum = 1
>>> sum
1
>>> type[sum]
>>> average = 1.23
>>> average
1.23
>>> average = 4.5e-6
>>> average
4.5e-06
>>> type[average]
>>> average = 78
>>> average
78
>>> type[average]
>>> msg = 'Hello'
>>> msg
'Hello'
>>> type[msg]
As mentioned, Python is dynamic typed. Python associates types with the objects, not the variables, i.e., a variable can hold object of any types, as shown in the above examples. Rules of Identifier [Names]An identifier starts with a letter [A-Z, a-z] or an underscore [_], followed by zero or more letters, underscores and digits [0-9].
Python does not allow special characters such as $ and @. KeywordsPython 3 has 35 reserved words, or keywords, which cannot be used as identifiers. - True, False, None [boolean and special literals]
- import, as, from
- if, elif, else, for, in, while, break, continue, pass, with [flow control]
- def, return, lambda, global, nonlocal [function]
- class
- and,
or, not, is, del [operators]
- try, except, finally, raise, assert [error handling]
- await, async, yield
Variable Naming ConventionA variable name is a noun, or a noun phrase made up of several words. There are two convenctions: - In lowercase words and optionally joined with underscore if it improves readability, e.g., num_students, x_max, myvar, isvalid, etc.
- In the so-called camel-case where
the first word is in lowercase, and the remaining words are initial-capitalized, e.g., numStudents, xMax, yMin, xTopLeft, isValidInput, and thisIsAVeryLongVariableName. [This is the Java's naming convention.]
Recommendations- It is important to choose a name that is self-descriptive and closely reflects the meaning of the variable, e.g., numStudents, but not n or x, to store the number of students. It is alright to use abbreviations, e.g., idx for index.
- Do
not use meaningless names like a, b, c, i, j, k, n, i1, i2, i3, j99, exercise85 [what is the purpose of this exercise?], and example12 [What is this example about?].
- Avoid single-letter names like i, j, k, a, b, c, which are easier to type but often meaningless. Exceptions are common names like x, y, z for coordinates, i for index.
Long names are harder to type, but self-document your program. [I suggest you spend sometimes practicing your typing.]
- Use singular and plural nouns prudently to differentiate between singular and plural variables. For example, you may use the variable row to refer to a single row number and the variable rows to refer to many rows [such as a list of rows - to be discussed later].
ConstantsPython does not support constants, where its
contents cannot be modified. [C supports constants via keyword const, Java via final.] It is a convention to name a variable in uppercase [joined with underscore], e.g., MAX_ROWS, SCREEN_X_MAX, to indicate that it should not be modified in the program. Nevertheless, nothing prevents it from being modified. Data Types: Number, String and ListPython supports various number type such as int [for integers such as 123, -456], float [for floating-point number
such as 3.1416, 1.2e3, -4.5E-6], and bool [for boolean of either True and False]. Python supports text string [a sequence of characters]. In Python, strings can be delimited with single-quotes or double-quotes, e.g., 'hello', "world", '' or "" [empty string]. Python supports a dynamic-array structure called list, denoted as lst = [v1, v2, ..., vn]. You can reference the i-th element as lst[i]. Python's list is similar to C/C++/Java's array, but it is NOT
fixed size, and can be expanded dynamically during runtime. I will describe these data types in details in the later section. Console Input/Output: input[] and print[] Built-in FunctionsYou can use built-in function input[] to read input from the console [as a string] and print[] to print output to the console. For example, >>> x = input['Enter a number: ']
Enter a number: 5
>>> x
'5'
>>> type[x]
>>> print[x]
5
>>> x = int[input['Enter an integer: ']]
Enter an integer: 5
>>> x
5
>>> type[x]
>>> print[x]
5print[]The built-in function print[] has the following signature: print[*objects, sep=' ', end='\n', file=sys.stdout, flush=False]
For
examples, >>> print['apple']
apple
>>> print['apple', 'orange']
apple orange
>>> print['apple', 'orange', 'banana']
apple orange bananaprint[]'s separator [sep] and ending [end]You can use the optional keyword-argument sep='x' to set the separator string [default is space], and end='x' for ending string [default is newline]. For examples, >>> for item in [1, 2, 3, 4]:
print[item]
1
2
3
4
>>> for item in [1, 2, 3, 4]:
print[item, end='']
1234
>>> for item in [1, 2, 3, 4]:
print[item, end='--']
1--2--3--4-->>> print['apple', 'orange', 'banana'] apple orange banana
>>> print['apple', 'orange', 'banana', sep=','] apple,orange,banana >>> print['apple', 'orange', 'banana', sep=':'] apple:orange:banana >>> print['apple', 'orange', 'banana', sep='|'] apple|orange|banana >>> print['apple', 'orange', 'banana', sep='\n'] apple orange bananaprint in Python 2 vs Python 3Recall that Python 2 and Python 3 are NOT compatible. In Python 2, you can use "print item", without the parentheses [because print is a
keyword in Python 2]. In Python 3, parentheses are required as print[] is a function. For example, >>> print['hello']
hello
>>> print 'hello'
File "", line 1
print 'hello'
^
SyntaxError: Missing parentheses in call to 'print'
>>> print['aaa', 'bbb']
aaa bbb
>>> print['Hello']
Hello
>>> print 'hello'
hello
>>> print['aaa', 'bbb']
['aaa', 'bbb']
>>> print 'aaa', 'bbb'
aaa bbb
Important: Always use print[] function with parentheses, for portability! Data Types and Dynamic TypingPython has a large number of built-in data types, such as Numbers [Integer, Float, Boolean, Complex Number], String, List, Tuple, Set, Dictionary and File. More high-level data types, such as Decimal and Fraction, are supported by external
modules. You can use the built-in function type[varName] to check the type of a variable or literal. Number TypesPython supports these built-in number types: - Integers [type int]: e.g., 123, -456. Unlike C/C++/Java, integers are of unlimited size in Python. For example, >>> 123 + 456 - 789
-210
>>> 123456789012345678901234567890 + 1
123456789012345678901234567891
>>> 1234567890123456789012345678901234567890 + 1
1234567890123456789012345678901234567891
>>> 2 ** 888
......
>>> len[str[2 ** 888]]
268
>>> type[123]
>>> help[int] You can also express integers in hexadecimal with prefix 0x [or 0X]; in octal with prefix 0o [or 0O]; and in binary with prefix 0b [or 0B]. For
examples, 0x1abc, 0X1ABC, 0o1776, 0b11000011.
- Floating-point numbers [type float]: e.g., 1.0, -2.3, 3.4e5, -3.4E-5, with a decimal point and an optional exponent [denoted by e or E]. floats are 64-bit double precision floating-point numbers. For example, >>> 1.23 * -4e5
-492000.0
>>> type[1.2]
>>> import math
>>> math.pi
3.141592653589793
>>> import random
>>> random.random[]
0.890839384187198
- Booleans [type bool]: takes a value of either True or False. Take note of the spelling in initial-capitalized. >>> 8 == 8
True
>>> 8 == 9
False
>>> type[True]
>>> type [8 == 8]
In Python, integer 0, an empty value [such as
empty string '', "", empty list [], empty tuple [], empty dictionary {}], and None are treated as False; anything else are treated as True. >>> bool[0]
False
>>> bool[1]
True
>>> bool['']
False
>>> bool['hello']
True
>>> bool[[]]
False
>>> bool[[1, 2, 3]]
True Booleans can also act as integers in arithmetic operations with 1 for True and 0 for False. For example, >>> True + 3
4
>>> False + 1
1 - Complex Numbers [type complex]: e.g., 1+2j, -3-4j. Complex numbers have a real part and an imaginary part denoted with suffix of j [or J]. For
example, >>> x = 1 + 2j
>>> x
[1+2j]
>>> x.real
1.0
>>> x.imag
2.0
>>> type[x]
>>> x * [3 + 4j]
[-5+10j]
- Other number types are provided by external modules, such as decimal module for decimal fixed-point numbers, fraction module for rational numbers. >>> 0.1 * 3
0.30000000000000004
>>> import decimal
>>> x = decimal.Decimal['0.1']
>>> x * 3
Decimal['0.3']
>>> type[x]
Dynamic Typing and Assignment OperatorRecall that Python is dynamic typed [instead of static typed]. Python associates types with objects, instead of variables. That is, a variable does not have a fixed type and can be assigned an object of any type. A
variable simply provides a reference to an object. You do not need to declare a variable before using a variable. A variable is created automatically when a value is first assigned, which links the assigned object to the variable. You can use built-in function type[var_name] to get the object type referenced by a variable. >>> x = 1
>>> x
1
>>> type[x]
>>> x = 1.0
>>> x
1.0
>>> type[x]
>>> x = 'hello'
>>> x
'hello'
>>> type[x]
>>> x = '123'
>>> x
'123'
>>> type[x]
Type Casting: int[x], float[x], str[x]You can perform
type conversion [or type casting] via built-in functions int[x], float[x], str[x], bool[x], etc. For example, >>> x = '123'
>>> type[x]
>>> x = int[x]
>>> x
123
>>> type[x]
>>> x = float[x]
>>> x
123.0
>>> type[x]
>>> x = str[x]
>>> x
'123.0'
>>> type[x]
>>> len[x]
5
>>> x = bool[x]
>>> x
True
>>> type[x]
>>> x = str[x]
>>> x
'True'
In summary, a variable does not associate with a type. Instead, a type is associated with an object. A variable provides a reference to an object [of a certain type]. Check Instance's Type: isinstance[instance, type]You can also use the built-in function isinstance[instance, type] to check if the instance belong to the type. For
example,
>>> isinstance[123, int]
True
>>> isinstance['a', int]
False
>>> isinstance['a', str]
TrueThe Assignment Operator [=]In Python, you do not need to declare variables before using the variables. The initial assignment creates a variable and links the assigned value to the variable. For example, >>> x = 8
>>> x = 'Hello'
>>> y
NameError: name 'y' is not definedPair-wise Assignment and Chain AssignmentFor example, >>> a = 1
>>> a
1
>>> b, c, d = 123, 4.5, 'Hello'
>>> b
123
>>> c
4.5
>>> d
'Hello'
>>> e = f = g = 123
>>> e
123
>>> f
123
>>> g
123Assignment operator is right-associative, i.e., a = b = 123 is interpreted as [a = [b = 123]]. del OperatorYou can use del operator to
delete a variable. For example, >>> x = 8
>>> x
8
>>> del x
>>> x
NameError: name 'x' is not definedNumber OperationsArithmetic Operators [+, -, *, /, //, **, %]Python supports these arithmetic operators:
OperatorModeUsageDescriptionExample +
| Binary
Unary
| x + y +x
| Addition
Positive
|
| -
| Binary
Unary
| x - y -x
| Subtraction
Negate
|
| *
| Binary
| x * y
| Multiplication
|
| /
| Binary
| x / y
| Float Division [Returns a float]
| 1 / 2 ⇒ 0.5 -1 / 2 ⇒ -0.5
| //
| Binary
| x // y
| Integer Division [Returns the floor integer]
| 1 // 2 ⇒ 0 -1 // 2 ⇒ -1 8.9 // 2.5 ⇒ 3.0 -8.9 // 2.5 ⇒ -4.0 [floor!] -8.9 // -2.5 ⇒ 3.0
| **
| Binary
| x ** y
| Exponentiation
| 2 ** 5 ⇒ 32 1.2 ** 3.4 ⇒ 1.858729691979481
| %
| Binary
| x % y
| Modulus [Remainder]
| 9 % 2 ⇒ 1 -9 % 2 ⇒ 1 9 % -2 ⇒ -1 -9 % -2 ⇒ -1 9.9 % 2.1 ⇒ 1.5 -9.9 % 2.1 ⇒ 0.6000000000000001
| Compound Assignment Operators [+=, -=, *=, /=, //=, **=, %=]Each of the arithmetic operators has a corresponding shorthand assignment counterpart, i.e., +=, -=, *=, /=, //=, **= and %=. For example i += 1 is the same as i = i + 1. Increment/Decrement [++, --]?Python does not
support increment [++] and decrement [--] operators [as in C/C++/Java]. You need to use i = i + 1 or i += 1 for increment. Python accepts ++i ⇒ +[+i] ⇒ i, and --i. Don't get trap into this. But Python flags a syntax error for i++ and i--. Mixed-Type OperationsFor mixed-type operations, e.g., 1 + 2.3 [int + float], the value of the "smaller" type is first promoted to the "bigger" type. It then performs the operation in the "bigger" type and returns the
result in the "bigger" type. In Python, int is "smaller" than float, which is "smaller" than complex. Relational [Comparison] Operators [==, !=, =, in, not in, is, is not]Python supports these relational [comparison] operators that return a bool value of either True or False.
OperatorModeUsageDescriptionExample == != =
| Binary
| x == y x != y x < y x y x >= y
| Comparison Return bool of either True or False
|
| in not in
| Binary
| x in seq x not in seq
| Check if x is contained in the sequence y Return bool of either True or False
| lst = [1, 2, 3] x = 1 x in lst ⇒ False
| is is not
| Binary
| x is y x is not y
| Check if x and y are referencing the same object Return bool of either True or False
|
|
Example: [TODO] Logical Operators [and, or, not]Python supports these logical [boolean] operators, that operate on boolean values.
OperatorModeUsageDescriptionExample and
| Binary
| x and y
| Logical AND
|
| or
| Binary
| x or y
| Logical OR
|
| not
| Unary
| not x
| Logical NOT
|
|
Notes: - Python's logical operators are typed out in word, unlike C/C++/Java which uses symbols &&, || and !.
- Python does not have an exclusive-or [xor] boolean operator.
Example: [TODO] Built-in FunctionsPython provides many built-in functions for numbers, including: - Mathematical functions: round[], pow[], abs[], etc.
- type[] to get the type.
- Type conversion
functions: int[], float[], str[], bool[], etc.
- Base radix conversion functions: hex[], bin[], oct[].
For examples, >>> x = 1.23456
>>> type[x]
>>> round[x]
1
>>> type[round[x]]
>>> round[x]
1.0
>>> type[round[x]]
>>> round[x, 1]
1.2
>>> round[x, 2]
1.23
>>> round[x, 8]
1.23456
>>> pow[2, 5]
32
>>> abs[-4.1]
4.1
>>> hex[1234]
'0x4d2'
>>> bin[254]
'0b11111110'
>>> oct[1234]
'0o2322'
>>> 0xABCD
43981
>>> dir[__built-ins__]
['type', 'round', 'abs', 'int', 'float', 'str', 'bool', 'hex', 'bin', 'oct',......]
>>> len[dir[__built-ins__]]
151
>>> len[dir[__built-ins__]]
144
>>> help[__built-ins__]
......Bitwise Operators [Advanced]Python supports these bitwise operators:
OperatorModeUsageDescriptionExample x=0b10000001 y=0b10001111 &
| binary
| x & y
| bitwise AND
| x & y ⇒ 0b10000001
| |
| binary
| x ! y
| bitwise OR
| x | y ⇒ 0b10001111
| ~
| Unary
| ~x
| bitwise NOT [or negate]
| ~x ⇒ -0b10000010
| ^
| binary
| x ^ y
| bitwise XOR
| x ^ y ⇒ 0b00001110
| count
| bitwise Right-Shift [padded with zeros]
| x >> 2 ⇒ 0b100000
|
StringIn Python, strings can be delimited by a pair of single-quotes ['...'] or double-quotes ["..."]. Python also supports multi-line strings via triple-single-quotes ['''...'''] or triple-double-quotes ["""..."""]. To place a single-quote ['] inside a single-quoted string, you need to use escape sequence \'. Similarly, to place a double-quote ["] inside a double-quoted string, use \". There is no need for escape sequence
to place a single-quote inside a double-quoted string; or a double-quote inside a single-quoted string. A triple-single-quoted or triple-double-quoted string can span multiple lines. There is no need for escape sequence to place a single/double quote inside a triple-quoted string. Triple-quoted strings are useful for multi-line documentation, HTML and other codes. Python 3 uses Unicode character set to support internationalization [i18n]. >>> s1 = 'apple'
>>> s1
'apple'
>>> s2 = "orange"
>>> s2
'orange'
>>> s3 = "'orange'"
>>> s3
"'orange'"
>>> s3 ="\"orange\""
>>> s3
'"orange"'
>>> s4 = """testing
12345"""
>>> s4
'testing\n12345'Escape Sequences for
Characters [\code]Like C/C++/Java, you need to use escape sequences [a back-slash + a code] for: - Special non-printable characters, such as tab [\t], newline [\n], carriage return [\r]
- Resolve ambiguity, such as \" [for " inside double-quoted string], \' [for ' inside single-quoted string], \\ [for \].
- \xhh for character in hex value and \ooo for octal value
- \uxxxx for
4-hex-digit [16-bit] Unicode character and \Uxxxxxxxx for 8-hex-digit [32-bit] Unicode character.
Raw Strings [r'...' or r"..."]You can prefix a string by r to disable the interpretation of escape sequences [i.e., \code], i.e., r'\n' is '\'+'n' [two characters] instead of newline [one character]. Raw strings are used extensively in regex [to be discussed in module re section]. Strings are ImmutableStrings are
immutable, i.e., their contents cannot be modified. String functions such as upper[], replace[] returns a new string object instead of modifying the string under operation.
Built-in Functions and Operators for StringsYou can operate on strings using: - built-in functions such as len[];
- operators such as in [contains], + [concatenation], * [repetition], indexing [i] and [-i], and slicing [m:n:step].
Note: These
functions and operators are applicable to all sequence data types including string, list, and tuple [to be discussed later].
Function / OperatorUsageDescriptionExamples s = 'Hello' len[]
| len[str]
| Length
| len[s] ⇒ 5
| in
| substr in str
| Contain? Return bool of either True or False
| 'ell' in s ⇒ True 'he' in s ⇒ False
| + +=
| str + str1 str += str1
| Concatenation
| s + '!' ⇒ 'Hello!'
| * *=
| str * count str *= count
| Repetition
| s * 2 ⇒ 'HelloHello'
| [i] [-i]
| str[i] str[-i]
| Indexing to get a character. The front index begins at 0; back index begins at -1 [=len[str]-1].
| s[1] ⇒ 'e' s[-4] ⇒ 'e'
| [m:n:step] [m:n] [m:] [:n] [:]
| str[m:n:step] str[m:n] str[m:] str[:n] str[:]
| Slicing to get a substring. From index m [included] to n [excluded] with step size. The defaults are: m=0, n=-1, step=1.
| s[1:3] ⇒ 'el' s[1:-2] ⇒ 'el' s[3:] ⇒ 'lo' s[:-2] ⇒ 'Hel' s[:] ⇒ 'Hello' s[0:5:2] ⇒ 'Hlo'
|
For examples, >>> s = "Hello, world"
>>> type[s]
>>> len[s]
12
>>> 'ello' in s
True
>>> s[0]
'H'
>>> s[1]
'e'
>>> s[-1]
'd'
>>> s[-2]
'l'
>>> s[1:3]
'el'
>>> s[1:-1]
'ello, worl'
>>> s[:4]
'Hell'
>>> s[4:]
'o, world'
>>> s[:]
'Hello, world'
>>> s = s + " again"
>>> s
'Hello, world again'
>>> s * 3
'Hello, world againHello, world againHello, world again'
>>> s = 'hello'
>>> print['The length of \"' + s + '\" is ' + len[s]]
TypeError: can only concatenate str [not "int"] to str
>>> print['The length of \"' + s + '\" is ' + str[len[s]]]
The length of "hello" is 5
>>> s[0] = 'a'
TypeError: 'str' object does not support item assignmentCharacter Type?Python does not have a dedicated character data type. A character is simply a string of length 1. You can use the indexing operator to extract individual character from a string, as shown in the above example; or process individual character using for-in loop [to be discussed later]. The built-in functions ord[] and chr[] operate on character, e.g., >>> ord['A']
65
>>> ord['水']
27700
>>> chr[65]
'A'
>>> chr[27700]
'水'Unicode vs ASCIIIn Python 3,
strings are defaulted to be Unicode. ASCII strings are represented as byte strings, prefixed with b, e.g., b'ABC'. In Python 2, strings are defaulted to be ASCII strings [byte strings]. Unicode strings are prefixed with u. You should always use Unicode for internationalization [i18n]! String-Specific Member FunctionsPython supports strings via a built-in class called str [We will describe class in the Object-Oriented Programming chapter]. The
str class provides many member functions. Since string is immutable, most of these functions return a new string. The commonly-used member functions are as follows, supposing that s is a str object: - s.strip[], s.rstrip[], s.lstrip[]: strip the leading and trailing whitespaces, the right [trailing] whitespaces; and the left [leading] whitespaces, respectively.
- s.upper[], s.lower[]: Return a uppercase/lowercase counterpart,
respectively.
- s.isupper[], s.islower[]: Check if the string is uppercase/lowercase, respectively.
- s.find[key_str]:
- s.index[key_str]:
- s.startswith[key_str]:
- s.endswith[key_str]:
- s.split[delimiter_str], delimiter_str.join[strings_list]:
>>> dir[str]
[..., 'capitalize', 'casefold', 'center', 'count', 'encode', 'endswith', 'expandtabs',
'find', 'format', 'format_map', 'index', 'isalnum', 'isalpha', 'isascii', 'isdecimal',
'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle',
'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'replace',
'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines',
'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
>>> s = 'Hello, world'
>>> type[s]
>>> dir[s]
.......
>>> help[s.find]
.......
>>> s.find['ll']
2
>>> s.find['app']
-1
>>> s.index['ll']
2
>>> s.index['app']
ValueError: substring not found
>>> s.startswith['Hell']
True
>>> s.endswith['world']
True
>>> s.replace['ll', 'xxx']
'Hexxxo, world'
>>> s.isupper[]
False
>>> s.upper[]
'HELLO, WORLD'
>>> s.split[', ']
['Hello', 'world']
>>> ', '.join[['hello', 'world', '123']]
'hello, world, 123'
>>> s = ' testing 12345 '
>>> s.strip[]
'testing 12345'
>>> s.rstrip[]
' testing 12345'
>>> s.lstrip[]
'testing 12345 '
>>> import string
>>> string.whitespace
' \t\n\r\x0b\x0c'
>>> string.digits
'0123456789'
>>> string.hexdigits
'0123456789abcdefABCDEF'String Formatting 1 [New Style]: Using str.format[] functionThere are a few ways to produce a formatted string for output. Python 3 introduces a new style in the str's format[] member function with
{} as place-holders [called format fields]. For examples, >>> '|{}|{}|more|'.format['Hello', 'world']
'|Hello|world|more|'
>>> '|{0}|{1}|more|'.format['Hello', 'world']
'|Hello|world|more|'
>>> '|{1}|{0}|more|'.format['Hello', 'world']
'|world|Hello|more|'
>>> '|{greeting}|{name}|'.format[greeting='Hello', name='Peter']
'|Hello|Peter|'
>>> '|{0}|{name}|more|'.format['Hello', name='Peter']
'|Hello|Peter|more|'
>>> '|{}|{name}|more|'.format['Hello', name='Peter']
'|Hello|Peter|more|'
>>> '|{1:8}|{0:7}|'.format['Hello', 'Peter']
'|Peter |Hello |'
>>> '|{1:8}|{0:>7}|{2:->> tup = ['a', 11, 22.22]
>>> tup = ['a', 11, 11.11]
>>> lst = ['b', 22, 22.22]
>>> '|{0[2]}|{0[1]}|{0[0]}|'.format[tup]
'|11.11|11|a|'
>>> '|{0[2]}|{0[1]}|{0[0]}|{1[2]}|{1[1]}|{1[0]}|'.format[tup, lst]
'|11.11|11|a|22.22|22|b|'
>>> dict = {'c': 33, 'cc': 33.33}
>>> '|{0[cc]}|{0[c]}|'.format[dict]
'|33.33|33|'
>>> '|{cc}|{c}|'.format[**dict]
'|33.33|33|'String Formatting 2: Using str.rjust[n], str.ljust[n], str.center[n],
str.zfill[n]You can also use str's member functions like str.rjust[n] [where n is the field-width], str.ljust[n], str.center[n], str.zfill[n] to format a string. For example, >>> '123'.rjust[5]
' 123'
>>> '123'.ljust[5]
'123 '
>>> '123'.center[5]
' 123 '
>>> '123'.zfill[5]
'00123'
>>> '1.2'.rjust[5]
' 1.2'
>>> '-1.2'.zfill[6]
'-001.2'String Formatting 3 [Old Style]: Using % operatorThe old style [in Python 2] is to use the % operator, with C-like printf[] format specifiers. For examples, >>> '|%s|%8s|%-8s|more|' % ['Hello', 'world', 'again']
'|Hello| world|again |more|'
>>> '|%d|%4d|%6.2f|' % [11, 222, 33.333]
'|11| 222| 33.33|'Avoid using old style for formatting. Conversion
between String and Number: int[], float[] and str[]You can use built-in functions int[] and float[] to parse a "numeric" string to an integer or a float; and str[] to convert a number to a string. For example, >>> s = '12345'
>>> s
'12345'
>>> type[s]
>>> i = int[s]
>>> i 12345 >>> type[i]
>>> s = '55.66'
>>> s
'55.66'
>>> f = float[s]
>>> f 55.66 >>> type[f]
>>> int[s]
ValueError: invalid literal for int[] with base 10: '55.66'
>>> i = 123
>>> s = str[i]
>>> s '123' >>> type[s]
'123'Concatenate a String and a Number?You CANNOT concatenate a string and a number [which results in TypeError]. Instead, you need to use the str[] function to convert the number to a string. For example, >>> 'Hello' + 123
TypeError: cannot concatenate 'str' and 'int' objects
>>> 'Hello' + str[123]
'Hello123'
The None ValuePython provides a special value called None [take note of the spelling in initial-capitalized], which can be used to initialize an object [to be discussed in OOP later]. For example, >>> x = None
>>> type[x]
>>> print[x]
None
>>> print[x is None]
True
>>> print[x is not None]
FalseList, Tuple, Dictionary and SetList [v1, v2,...]Python has a powerful built-in dynamic array called list. - A list is enclosed by square brackets [].
- A list can
contain items of different types. It is because Python associates types to objects, not variables.
- A list grows and shrinks in size automatically [dynamically]. You do not have to specify its size during initialization.
- A list is mutable. You can update its contents.
Built-in Functions and Operators for listA list, like string, is a sequence. Hence, you can operate lists using: - built-in sequence functions such as
len[].
- built-in sequence functions for list of numbers such as max[], min[], and sum[].
- built-in operators such as in [contains], + [concatenation] and * [repetition], del, [i] [indexing], and [m,n,step] [slicing].
Notes: - You can index the items from the front with positive index, or from the back with negative index. E.g., if lst is a list, lst[0] and lst[1] refer to its first and second items; lst[-1]
and lst[-2] refer to the last and second-to-last items.
- You can also refer to a sub-list [or slice] using slice notation lst[m:n:step] [from index m [included] to index n [excluded] with step size].
OperatorUsageDescriptionExamples lst = [8, 9, 6, 2] in not in
| x in lst x not in lst
| Contain? Return bool of either True or False
| 9 in lst ⇒ True 5 in lst ⇒ False
| + +=
| lst + lst1 lst += lst1
| Concatenation
| lst + [5, 2] ⇒ [8, 9, 6, 2, 5, 2]
| * *=
| lst * count lst *= count
| Repetition
| lst * 2 ⇒ [8, 9, 6, 2, 8, 9, 6, 2]
| [i] [-i]
| lst[i] lst[-i]
| Indexing to get an item. Front index begins at 0; back index begins at -1 [or len[lst]-1].
| lst[1] ⇒ 9 lst[-2] ⇒ 6
| [m:n:step] [m:n] [m:] [:n] [:]
| lst[m:n:step] lst[m:n] lst[m:] lst[:n] lst[:]
| Slicing to get a sublist. From index m [included] to n [excluded] with step size. The defaults are: m is 0, n is len[lst]-1.
| lst[1:3] ⇒ [9, 6] lst[1:-2] ⇒ [9] lst[3:] ⇒ [2] lst[:-2] ⇒ [8, 9] lst[:] ⇒ [8, 9, 6, 2] lst[0:4:2] ⇒ [8, 6] newlst = lst[:] ⇒ Copy lst[4:] = [1, 2] ⇒ Extend
| del
| del lst[i] del lst[m:n] del lst[m:n:step]
| Delete one or more items
| del lst[1] ⇒ [8, 6, 2] del lst[1:] ⇒ [8] del lst[:] ⇒ [] [Clear]
|
FunctionUsageDescriptionExamples lst = [8, 9, 6, 2] len[]
| len[lst]
| Length
| len[lst] ⇒ 4
| max[] min[]
| max[lst] min[lst]
| Maximum value minimum value
| max[lst] ⇒ 9 min[lst] ⇒ 2
| sum[]
| sum[lst]
| Sum [for number lists only]
| sum[lst] ⇒ 16
|
list, unlike string, is mutable. You can insert, remove and modify its items. For examples, >>> lst = [123, 4.5, 'hello', True, 6+7j]
>>> lst
[123, 4.5, 'hello', True, [6+7j]]
>>> len[lst]
5
>>> type[lst]
>>> lst[0]
123
>>> lst[-1]
[6+7j]
>>> lst[2] = 'world'
>>> lst
[123, 4.5, 'world', True, [6+7j]]
>>> lst[0:2]
[123, 4.5]
>>> lst[:3]
[123, 4.5, 'world']
>>> lst[2:]
['world', True, [6+7j]]
>>> lst[::2]
[123, 'world']
>>> lst[::-1]
['world', 4.5, 123]
>>> lst[2:4]
['world', True]
>>> lst[2:4] = 0
TypeError: can only assign an iterable
>>> lst[2:4] = [1, 2, 'a', 'b']
>>> lst
[123, 4.5, 1, 2, 'a', 'b', [6+7j]]
>>> lst[1:3] = []
>>> lst
[123, 2, 'a', 'b', [6+7j]]
>>> lst[::2] = ['x', 'y', 'z']
>>> lst
['x', 2, 'y', 'b', 'z']
>>> lst[::2] = [1, 2, 3, 4]
ValueError: attempt to assign sequence of size 4 to extended slice of size 3
>>> 'x' in lst
True
>>> 'a' in lst
False
>>> lst + [6, 7, 8]
['x', 2, 'y', 'b', 'z', 6, 7, 8]
>>> lst * 3
['x', 2, 'y', 'b', 'z', 'x', 2, 'y', 'b', 'z', 'x', 2, 'y', 'b', 'z']
>>> del lst[1]
>>> lst
['x', 'y', 'b', 'z']
>>> del lst[::2]
>>> lst
['y', 'z']
>>> lst = [123, 4.5, ['a', 'b', 'c']]
>>> lst
[123, 4.5, ['a', 'b', 'c']]
>>> lst[2]
['a', 'b', 'c']Appending Items to a list>>> lst = [123, 'world']
>>> lst[2]
IndexError: list index out of range
>>> lst[len[lst]] = 4.5
IndexError: list assignment index out of range
>>> lst[len[lst]:] = [4.5]
>>> lst
[123, 'world', 4.5]
>>> lst[len[lst]:] = [6, 7, 8]
>>> lst
[123, 'world', 4.5, 6, 7, 8]
>>> lst.append['nine']
>>> lst
[123, 'world', 4.5, 6, 7, 8, 'nine']
>>> lst.extend[['a', 'b']]
>>> lst
[123, 'world', 4.5, 6, 7, 8, 'nine', 'a', 'b']
>>> lst + ['c']
[123, 'world', 4.5, 6, 7, 8, 'nine', 'a', 'b', 'c']
>>> lst
[123, 'world', 4.5, 6, 7, 8, 'nine', 'a', 'b']Copying a list>>> l1 = [123, 4.5, 'hello']
>>> l2 = l1[:]
>>> l2
[123, 4.5, 'hello']
>>> l2[0] = 8
>>> l2
[8, 4.5, 'hello']
>>> l1
[123, 4.5, 'hello']
>>> l3 = l1.copy[]
>>> l4 = l1
>>> l4
[123, 4.5, 'hello']
>>> l4[0] = 8
>>> l4
[8, 4.5, 'hello']
>>> l1
[8, 4.5, 'hello']list-Specific Member FunctionsThe list class provides many member functions. Suppose lst is a list object: - lst.index[item]: return the index of the first occurrence of item; or error.
- lst.append[item]: append the given item behind the lst and return None; same as slicing operation lst[len[lst]:] = [item].
- lst.extend[lst1]: append the given list lst1 behind the lst and return None; same as slicing operation lst[len[lst]:] = lst1.
- lst.insert[index, item]: insert the given item before the index and return None. Hence, lst.insert[0, item] inserts before the first item of the lst; lst.insert[len[lst], item] inserts at the end of the lst which is the same as lst.append[item].
- lst.remove[item]:
remove the first occurrence of item from the lst and return None; or error.
- lst.pop[]: remove and return the last item of the lst.
- lst.pop[index]: remove and return the indexed item of the lst.
- lst.clear[]: remove all the items from the lst and return None; same as operator del lst[:].
- lst.count[item]: return the occurrences of item.
- lst.reverse[]: reverse the lst in place and return None.
- lst.sort[]: sort the lst
in place and return None.
- lst.copy[]: return a copy of lst; same as lst[:].
Recall that list is mutable [unlike string which is immutable]. These functions modify the list directly. For examples, >>> lst = [123, 4.5, 'hello', [6, 7, 8]]
>>> lst
[123, 4.5, 'hello', [6, 7, 8]]
>>> type[lst]
>>> dir[lst]
>>> len[lst]
4
>>> lst.append['apple']
>>> lst
[123, 4.5, 'hello', [6, 7, 8], 'apple']
>>> len[lst]
5
>>> lst.pop[1]
4.5
>>> lst
[123, 'hello', [6, 7, 8], 'apple']
>>> len[lst]
4
>>> lst.insert[2, 55.66]
>>> lst
[123, 'hello', 55.66, [6, 7, 8], 'apple']
>>> del lst[3:]
>>> lst
[123, 'hello', 55.66]
>>> lst.append[55.66]
>>> lst
[123, 'hello', 55.66, 55.66]
>>> lst.remove[55.66]
>>> lst
[123, 'hello', 55.66]
>>> lst.reverse[]
>>> lst
[55.66, 'hello', 123]
>>> lst2 = [5, 8, 2, 4, 1]
>>> lst2.sort[]
>>> lst2
[1, 2, 4, 5, 8]
>>> lst2.index[5]
3
>>> lst2.index[9]
......
ValueError: 9 is not in list
>>> lst2.append[1]
>>> lst2
[1, 2, 4, 5, 8, 1]
>>> lst2.count[1]
2
>>> lst2.count[9]
0
>>> sorted[lst2]
[1, 1, 2, 4, 5, 8]
>>> lst2
[1, 2, 4, 5, 8, 1] Using list as a last-in-first-out StackTo use a list as a last-in-first-out [LIFO] stack, use append[item] to add an item to the top-of-stack [TOS] and pop[] to remove the item from the TOS. Using list as a
first-in-first-out QueueTo use a list as a first-in-first-out [FIFO] queue, use append[item] to add an item to the end of the queue and pop[0] to remove the first item of the queue. However, pop[0] is slow! The standard library provide a class collections.deque to efficiently implement deque with fast appends and pops from both ends. Tuple [v1, v2,...]Tuple is similar to list except that it is immutable [just like string]. Hence, tuple is more
efficient than list. A tuple consists of items separated by commas, enclosed in parentheses []. >>> tup = [123, 4.5, 'hello']
>>> tup
[123, 4.5, 'hello']
>>> tup[1]
4.5
>>> tup[1:3]
[4.5, 'hello']
>>> tup[1] = 9
TypeError: 'tuple' object does not support item assignment
>>> type[tup]
>>> lst = list[tup] >>> lst
[123, 4.5, 'hello']
>>> type[lst]
An one-item tuple needs a comma to differentiate from parentheses: >>> tup = [5,]
>>> tup
[5,]
>>> x = [5]
>>> x
5The parentheses are actually optional, but recommended for readability. Nevertheless, the commas are mandatory. For example, >>> tup = 123, 4.5, 'hello'
>>> tup
[123, 4.5, 'hello']
>>> tup2 = 88,
>>> tup2
[88,]
>>> tup3 = []
>>> tup3
[]
>>> len[tup3]
0You can operate on tuples using [supposing that tup is a tuple]: - built-in functions such as len[tup];
- built-in functions for tuple of numbers
such as max[tup], min[tup] and sum[tup];
- operators such as in, + and *; and
- tuple's member functions such as tup.count[item], tup.index[item], etc.
Conversion between List and TupleYou can covert a list to a tuple using built-in function tuple[]; and a tuple to a list using list[]. For examples, >>> tuple[[1, 2, 3, 1]]
[1, 2, 3, 1]
>>> list[[1, 2, 3, 1]]
[1, 2, 3, 1]Dictionary {k1:v1, k2:v2,...}Python's built-in dictionary type supports key-value pairs [also known
as name-value pairs, associative array, or mappings]. - A dictionary is enclosed by a pair of curly braces {}. The key and value are separated by a colon [:], in the form of {k1:v1, k2:v2, ...}
- Unlike list and tuple, which index items using an integer index 0, 1, 2, 3,..., dictionary can be indexed using any key type, including number, string or other types.
- Dictionary is mutable.
>>> dct = {'name':'Peter', 'gender':'male', 'age':21}
>>> dct
{'age': 21, 'name': 'Peter', 'gender': 'male'}
>>> dct['name']
'Peter'
>>> dct['age'] = 22
>>> dct
{'age': 22, 'name': 'Peter', 'gender': 'male'}
>>> len[dct]
3
>>> dct['email'] = ''
>>> dct
{'name': 'Peter', 'age': 22, 'email': '', 'gender': 'male'}
>>> type[dct]
>>> dct2 = dict[[['a', 1], ['c', 3], ['b', 2]]]
>>> dct2
{'b': 2, 'c': 3, 'a': 1}Dictionary-Specific Member
FunctionsThe dict class has many member methods. The commonly-used are follows [suppose that dct is a dict object]: - dct.has_key[]:
- dct.items[], dct.keys[], dct.values[]:
- dct.clear[]:
- dct.copy[]:
- dct.get[]:
- dct.update[dct2]: merge the given dictionary dct2 into dct. Override the value if key exists, else, add new key-value.
- dct.pop[]:
For Examples, >>> dct = {'name':'Peter', 'age':22, 'gender':'male'}
>>> dct
{'gender': 'male', 'name': 'Peter', 'age': 22}
>>> type[dct]
>>> dir[dct]
......
>>> list[dct.keys[]]
['gender', 'name', 'age']
>>> list[dct.values[]]
['male', 'Peter', 22]
>>> list[dct.items[]]
[['gender', 'male'], ['name', 'Peter'], ['age', 22]]
>>> dct.get['age', 'not such key']
22
>>> dct.get['height', 'not such key']
'not such key'
>>> dct['height']
KeyError: 'height'
>>> del dct['age']
>>> dct
{'gender': 'male', 'name': 'Peter'}
>>> 'name' in dct
True
>>> dct.update[{'height':180, 'weight':75}]
>>> dct
{'height': 180, 'gender': 'male', 'name': 'Peter', 'weight': 75}
>>> dct.pop['gender']
'male'
>>> dct
{'name': 'Peter', 'weight': 75, 'height': 180}
>>> dct.pop['no_such_key']
KeyError: 'no_such_key'
>>> dct.pop['no_such_key', 'not found']
'not found'Set {k1, k2,...}A set is an
unordered, non-duplicate collection of objects. A set is delimited by curly braces {}, just like dictionary. You can think of a set as a collection of dictionary keys without associated values. Sets are mutable. For example, >>> st = {123, 4.5, 'hello', 123, 'Hello'}
>>> st
{'Hello', 'hello', 123, 4.5}
>>> 123 in st
True
>>> 88 in st
False
>>> st2 = set[[2, 1, 3, 1, 3, 2]]
>>> st2
{1, 2, 3}
>>> st3 = set['hellllo']
>>> st3
{'o', 'h', 'e', 'l'}
Set-Specific Operators [&, !, -, ^]Python supports set operators & [intersection], | [union], - [difference] and ^ [exclusive-or]. For example, >>> st1 = {'a', 'e', 'i', 'o', 'u'}
>>> st1
{'e', 'o', 'u', 'a', 'i'}
>>> st2 = set['hello']
>>> st2
{'o', 'l', 'e', 'h'}
>>> st1 & st2
{'o', 'e'}
>>> st1 | st2
{'o', 'l', 'h', 'i', 'e', 'a', 'u'}
>>> st1 - st2
{'i', 'u', 'a'}
>>> st1 ^ st2
{'h', 'i', 'u', 'a', 'l'}Sequence
Types: list, tuple, strlist, tuple, and str are parts of the sequence types. list is mutable, while tuple and str are immutable. They share the common sequence's built-in operators and built-in functions, as follows:
Opr / FuncUsageDescription in not in
| x in seq x not in seq
| Contain? Return bool of either True or False
| +
| seq + seq1
| Concatenation
| *
| seq * count
| Repetition [Same as: seq + seq + ...]
| [i] [-i]
| seq[i] seq[-i]
| Indexing to get an item. Front index begins at 0; back index begins at -1 [or len[seq]-1].
| [m:n:step] [m:n] [m:] [:n] [:]
| seq[m:n:step] seq[m:n] seq[m:] seq[:n} seq[:]
| Slicing to get a sub-sequence. From index m [included] to n [excluded] with step size. The defaults are: m is 0, n is len[seq]-1.
| len[] min[] max[]
| len[seq] min[seq] max[seq]
| Return the Length, mimimum and maximum of the sequence
| seq.index[]
| seq.index[x] seq.index[x, i] seq.index[x, i, j]
| Return the index of x in the sequence, or raise ValueError. Search from i [included] to j [excluded]
| seq.count[]
| seq.count[x]
| Returns the count of x in the sequence
|
For mutable sequences [list], the following built-in operators and built-in functions [func[seq]] and member functions [seq.func[*args]] are supported:
Opr / FuncUsageDescription []
| seq[i] = x seq[m:n] = [] seq[:] = [] seq[m:n] = seq1 seq[m:n:step] = seq1
| Replace one item Remove one or more items Remove all items Replace more items with a sequence of the same size
| +=
| seq += seq1
| Extend by seq1
| *=
| seq *= count
| Repeat count times
| del
| del seq[i] del seq[m:n] del seq[m:n:step]
| Delete one item Delete more items, same as: seq[m:n] = []
| seq.clear[]
| seq.clear[]
| Remove all items, same as: seq[:] = [] or del seq[:]
| seq.append[]
| seq.append[x]
| Append x to the end of the sequence, same as: seq[len[seq]:len[seq]] = [x]
| seq.extend[]
| seq.entend[seq1]
| Extend the sequence, same as: seq[len[seq]:len[seq]] = seq1 or seq += seq1
| seq.insert[]
| seq.insert[i, x]
| Insert x at index i, same as: seq[i] = x
| seq.remove[]
| seq.remove[x]
| Remove the first occurence of x
| seq.pop[]
| seq.pop[] seq.pop[i]
| Retrieve and remove the last item Retrieve and remove the item at index i
| seq.copy[]
| seq.copy[]
| Create a shallow copy of seq, same as: seq[:]
| seq.reverse[]
| seq.reverse[]
| Reverse the sequence in place
|
OthersDeque[TODO] Heap[TODO] Flow Control ConstructsConditional if-elif-elseThe syntax is as follows. The elif [else-if] and else blocks are optional. if test:
true_block
else:
false_block
if test_1:
block_1
elif test_2:
block_2
elif test_3:
block_3
......
......
elif test_n:
block_n
else:
else_blockFor example: if x == 0:
print['x is zero']
elif x > 0:
print['x is more than zero']
print['xxxx']
else:
print['x is less than zero']
print['yyyy']There is no switch-case statement in Python [as in C/C++/Java]. Comparison and Logical OperatorsPython supports these comparison [relational] operators, which return a
bool of either True or False. - < [less than], [greater than], >= [greater than or equal to]. [This is the same as C/C++/Java.]
- in, not in: Check if an item is|is not in a sequence [list, tuple, string, set, etc].
- is, is not: Check if two variables have the same reference.
Python supports these logical [boolean] operators:
and, or, not. [C/C++/Java uses &&, ||, !.] Chain Comparison v1 < x < v2Python supports chain comparison in the form of v1 < x < v2, e.g., >>> x = 8
>>> 1 < x < 10
True
>>> 1 < x and x < 10
True
>>> 10 < x < 20
False
>>> 10 > x > 1
True
>>> not [10 < x < 20]
TrueComparing SequencesThe comparison operators [such as ==, >> 'a' < 'b'
True
>>> 'ab' < 'aa'
False
>>> 'a' < 'b' < 'c'
True
>>> [1, 2, 3] < [1, 2, 4]
True
>>> [1, 2, 3] >> [1, 2, 3] < [1, 2, 3] FalseShorthand if-else [or Conditional Expression]The syntax is: true_expr if test else false_expr
For example, >>> x = 0
>>> print['zero' if x == 0 else 'not zero']
zero
>>> x = -8
>>> abs_x = x if x > 0 else -x
>>> abs_x
8Note: Python does not use "? :" for shorthand if-else, as in C/C++/Java. The while loopThe syntax is as follows: while test:
true_block
while test:
true_block
else:
else_blockThe else block is optional, which will be executed if the loop exits normally without encountering a break statement. For example, upperbound = int[input['Enter the upperbound: ']]
sum = 0
number = 1
while number >> for char in 'hello': print[char]
h
e
l
l
o
>>> for item in [123, 4.5, 'hello']: print[item]
123
4.5
hello
>>> for item in [123, 4.5, 'hello']: print[item]
123
4.5
hello
>>> dct = {'a': 1, 2: 'b', 'c': 'cc'}
>>> for key in dct: print[key, ':', dct[key]]
a : 1
c : cc
2 : b
>>> for item in {'apple', 1, 2, 'apple'}: print[item]
1
2
apple
>>> infile = open['test.txt', 'r']
>>> for line in infile: print[line]
...Each line of the file...
>>> infile.close[]
for[;;] LoopPython does NOT support the
C/C++/Java-like for[int i; i < n; ++i] loop, which uses a varying index for the iterations. Take note that you cannot use the "for item in lst" loop to modify a list. To modify the list, you need to get the list indexes by creating an index list. For example, >>> lst = [11, 22, 33]
>>> for item in lst:
item += 1
>>> print[lst]
[11, 22, 33]
>>> idx_lst = [0, 1, 2]
>>> for idx in idx_lst:
lst[idx] += 1
>>> print[lst]
[12, 23, 34]Manually creating the index list is not practical. You can use the range[] function to create the index list [described below]. The range[] Built-in FunctionThe range[] function produces a series of
running integers, which can be used as index list for the for-in loop. - range[n] produces integers from 0 to n-1;
- range[m, n] produces integers from m to n-1;
- range[m, n, s] produces integers from m to n-1 in step of s.
For example, upperbound = int[input['Enter the upperbound: ']]
sum = 0
for number in range[1, upperbound+1]:
sum += number
print['The sum is:', sum]
lst = [9, 8, 4, 5]
sum = 0
for idx in range[len[lst]]:
sum += lst[idx]
print['The sum is:', sum]
lst = [9, 8, 4, 5]
sum = 0
for item in lst:
sum += item
print['The sum is:', sum]
del sum
print['The sum is:', sum[lst]]
for idx in range[len[lst]]:
lst[idx] += 1
print[lst]
idx = 0
while idx < len[lst]:
lst[idx] += 1
idx += 1
print[lst]
lst = [11, 22, 33]
lst1 = [item + 1 for item in lst]
print[lst1]Using else-clause in LoopRecall that the else-clause will be executed only if the loop exits without encountering a break. for number in range[2, 101]:
for factor in range[2, number//2+1]:
if number % factor == 0:
print['{} is NOT a prime'.format[number]]
break
else:
print['{} is a prime'.format[number]] Iterating
through a Sequence of SequencesA sequence [such as list, tuple] can contain sequences. For example, >>> lst = [[1,'a'], [2,'b'], [3,'c']]
>>> for v1, v2 in lst: print[v1, v2]
1 a
2 b
3 c
>>> lst = [[1, 2, 3], ['a', 'b', 'c']]
>>> for v1, v2, v3 in lst: print[v1, v2, v3]
1 2 3
a b cIterating through a DictionaryThere are a few ways to iterate through an dictionary: >>> dct = {'name':'Peter', 'gender':'male', 'age':21}
>>> for key in dct: print[key, ':', dct[key]]
age : 21
name : Peter
gender : male
>>> for key, value in dct.items[]: print[key, ':', value]
age : 21
name : Peter
gender : male
>>> dct.items[]
[['gender', 'male'], ['age', 21], ['name', 'Peter']]The iter[] and next[] Built-in FunctionsThe built-in function iter[iterable] takes a iterable [such as sequence] and returns an iterator object. You can then use next[iterator] to iterate through the items. For example, >>> lst = [11, 22, 33]
>>> iterator = iter[lst]
>>> next[iterator]
11
>>> next[iterator]
22
>>> next[iterator]
33
>>> next[iterator]
StopIteration
>>> type[iterator]
The
reversed[] Built-in FunctionTo iterate a sequence in the reverse order, apply the reversed[] function which reverses the iterator over values of the sequence. For example, >>> lst = [11, 22, 33]
>>> for item in reversed[lst]: print[item, end=' ']
33 22 11
>>> reversed[lst]
>>> str = "hello"
>>> for ch in reversed[str]: print[ch, end='']
ollehThe enumerate[] Built-in FunctionYou can use the built-in function enumerate[] to obtain the positional indexes, when looping through a sequence. For example, >>> lst = ['a', 'b', 'c']
>>> for idx, value in enumerate[lst]: print[idx, value]
0 a
1 b
2 c
>>> enumerate[lst]
>>> lst = [11, 22, 33]
>>> for idx, value in enumerate[lst]: lst[idx] += 1
>>> lst
[12, 23, 34]
>>> tup = ['d', 'e', 'f']
>>> for idx, value in enumerate[tup]: print[idx, value]
0 d
1 e
2 fMultiple Sequences and the zip[] Built-in FunctionTo loop over two or more
sequences concurrently, you can pair the entries with the zip[] built-in function. For examples, >>> lst1 = ['a', 'b', 'c']
>>> lst2 = [11, 22, 33]
>>> for i1, i2 in zip[lst1, lst2]: print[i1, i2]
a 11
b 22
c 33
>>> zip[lst1, lst2]
[['a', 11], ['b', 22], ['c', 33]]
>>> tuple3 = [44, 55]
>>> zip[lst1, lst2, tuple3]
[['a', 11, 44], ['b', 22, 55]]Comprehension for Generating Mutable List, Dictionary and SetList comprehension provides concise way to generate a new list. The syntax is: result_list = [expression_with_item for item in list]
result_list = [expression_with_item for item in list if test]
result_list = []
for item in list:
if test:
result_list.append[item]For examples, >>> sq_lst = [item * item for item in range[1, 11]]
>>> sq_lst
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
>>> sq_lst = []
>>> for item in range[1, 11]:
sq_lst.append[item * item]
>>> lst = [3, 4, 1, 5]
>>> sq_lst = [item * item for item in lst]
>>> sq_lst
[9, 16, 1, 25]
>>> sq_lst_odd = [item * item for item in lst if item % 2 != 0]
>>> sq_lst_odd
[9, 1, 25]
>>> sq_lst_odd = []
>>> for item in lst:
if item % 2 != 0:
sq_lst_odd.append[item * item]
>>> lst = [[x, y] for x in range[1, 3] for y in range[1, 4] if x != y]
>>> lst
[[1, 2], [1, 3], [2, 1], [2, 3]]
>>> lst = []
>>> for x in range[1,3]:
for y in range[1,4]:
if x != y: lst.append[[x, y]]
>>> lst
[[1, 2], [1, 3], [2, 1], [2, 3]]Similarly, you can create dictionary and set [mutable sequences] via comprehension. For example, >>> dct = {x:x**2 for x in range[1, 5]}
>>> dct
{1: 1, 2: 4, 3: 9, 4: 16}
>>> set = {ch for ch in 'hello' if ch not in 'aeiou'}
>>> set
{'h', 'l'}Comprehension cannot be used to generate string and tuple, as they are
immutable and append[] cannot be applied. Naming Conventions and Coding Styles [PEP 8 & PEP 257]Naming ConventionsThese are the recommended naming conventions in Python: - Variable names: use a noun in lowercase words [optionally joined with underscore if it improves readability], e.g., num_students.
- Function names: use a verb in lowercase words [optionally joined with underscore if it improves readability], e.g., getarea[] or get_area[].
- Class names: use a noun in camel-case [initial-cap all words], e.g., MyClass, IndexError, ConfigParser.
- Constant names: use a noun in uppercase words joined with underscore, e.g., PI, MAX_STUDENTS.
Coding StylesRead: - "PEP 8: Style Guide for Python Code"
- "PEP 257: Doc-string Conventions"
The
recommended styles are: - Use 4 spaces for indentation. Don't use tab.
- Lines shall not exceed 79 characters.
- Use blank lines to separate functions and classes.
- Use a space before and after an operator.
- [TODO] more
FunctionsSyntaxIn Python, you define a function via the keyword def followed by the function name, the parameter list, the doc-string and the function body. Inside the function body, you can use
a return statement to return a value to the caller. There is no need for type declaration like C/C++/Java. The syntax is: def function_name[arg1, arg2, ...]:
"""Function doc-string"""
body_block
return return-valueExample 1>>> def my_square[x]:
"""Return the square of the given number"""
return x * x
>>> my_square[8]
64
>>> my_square[1.8]
3.24
>>> my_square['hello']
TypeError: can't multiply sequence by non-int of type 'str'
>>> my_square
>>> type[my_square]
>>> my_square.__doc__
'Return the square of the given number'
>>> help[my_square]
my_square[x]
Return the square of the given number
>>> dir[my_square]
[......]Take note that you need to define the function before using it, because Python is interpretative. Example 2def fibon[n]:
"""Print the first n Fibonacci numbers, where f[n]=f[n-1]+f[n-2] and f[1]=f[2]=1"""
a, b = 1, 1
for count in range[n]:
print[a, end=' ']
a, b = b, a+b
print[]
fibon[20]Example 3: Function doc-stringdef my_cube[x]:
return x*x*x
print[my_cube[8]]
print[my_cube[-8]]
print[my_cube[0]] This example elaborates on the function's doc-string: - The first line "[number] -> [number]" specifies the type of the argument and return value. Python does
not perform type check on function, and this line merely serves as documentation.
- The second line gives a description.
- Examples of function invocation follow. You can use the doctest module to perform unit test for this function based on these examples [to be described in the "Unit Test" section.
The pass statementThe pass statement does nothing. It is sometimes needed as a dummy statement placeholder to ensure correct syntax, e.g., def my_fun[]:
pass
Function Parameters and ArgumentsPassing Arguments by Value vs. by ReferenceIn Python: - Immutable arguments [such as integers, floats, strings and tuples] are passed by value. That is, a copy is cloned and passed into the function. The original cannot be modified inside the function.
- Mutable arguments [such as lists, dictionaries, sets and instances of classes] are passed by reference. That is, they can be modified inside the function.
For examples, >>> def increment_int[number]:
number += 1
>>> number = 5
>>> increment_int[number]
>>> number
5
>>> def increment_list[lst]:
for i in range[len[lst]]:
lst[i] += lst[i]
>>> lst = [1, 2, 3, 4, 5]
>>> increment_list[lst]
>>> lst
[2, 4, 6, 8, 10] Function Parameters with Default ValuesYou can assign a default value to the "trailing" function parameters. These trailing parameters having default values are optional during invocation. For example, >>> def my_sum[n1, n2 = 4, n3 = 5]:
"""Return the sum of all the arguments"""
return n1 + n2 + n3
>>> print[my_sum[1, 2, 3]]
6
>>> print[my_sum[1, 2]]
8
>>> print[my_sum[1]]
10
>>> print[my_sum[]]
TypeError: my_sum[] takes at least 1 argument [0 given]
>>> print[my_sum[1, 2, 3, 4]]
TypeError: my_sum[] takes at most 3 arguments [4 given]Another example, def greet[name]:
return 'hello, ' + name
greet['Peter'] In stead of hard-coding the 'hello, ', it is more flexible to use a parameter with a default value, as follows: def greet[name, prefix='hello']:
return prefix + ', ' + name
greet['Peter']
greet['Peter', 'hi']
greet['Peter', prefix='hi']
greet[name='Peter', prefix='hi']
Positional and Keyword ArgumentsPython functions support both
positional and keyword [or named] arguments. Normally, Python passes the arguments by position from left to right, i.e., positional, just like C/C++/Java. Python also allows you to pass arguments by keyword [or name] in the form of kwarg=value. For example, def my_sum[n1, n2 = 4, n3 = 5]:
"""Return the sum of all the arguments"""
return n1 + n2 + n3
print[my_sum[n2 = 2, n1 = 1, n3 = 3]]
print[my_sum[n2 = 2, n1 = 1]]
print[my_sum[n1 = 1]]
print[my_sum[1, n3 = 3]]
print[my_sum[n2 = 2]] # TypeError, n1 missingYou can also mix the positional arguments and keyword arguments, but you need to place the positional arguments first, as shown in the above examples. Variable Number of Positional Parameters
[*args]Python supports variable [arbitrary] number of arguments. In the function definition, you can use * to pack all the remaining positional arguments into a tuple. For example, def my_sum[a, *args]:
"""Return the sum of all the arguments [one or more]"""
sum = a
print['args is:', args]
for item in args:
sum += item
return sum
print[my_sum[1]]
print[my_sum[1, 2]]
print[my_sum[1, 2, 3]]
print[my_sum[1, 2, 3, 4]] Python supports placing *args in the middle of the parameter list. However, all the arguments after *args must be passed by keyword to avoid ambiguity. For example def my_sum[a, *args, b]:
sum = a
print['args is:', args]
for item in args:
sum += item
sum += b
return sum
print[my_sum[1, 2, 3, 4]]
# TypeError: my_sum[] missing 1 required keyword-only argument: 'b'
print[my_sum[1, 2, 3, 4, b=5]] Unpacking List/Tuple into Positional Arguments [*lst,
*tuple]In the reverse situation when the arguments are already in a list/tuple, you can also use * to unpack the list/tuple as separate positional arguments. For example, >>> def my_sum[a, b, c]:
return a+b+c
>>> lst = [11, 22, 33]
>>> my_sum[lst]
TypeError: my_sum[] missing 2 required positional arguments: 'b' and 'c'
>>> my_sum[*lst]
66
>>> lst = [44, 55]
>>> my_sum[*lst]
TypeError: my_sum[] missing 1 required positional argument: 'c'
>>> def my_sum[*args]:
sum = 0
for item in args: sum += item
return sum
>>> my_sum[11, 22, 33]
66
>>> lst = [44, 55, 66]
>>> my_sum[*lst]
165
>>> tup = [7, 8, 9, 10]
>>> my_sum[*tup]
34Variable Number of Keyword Parameters [**kwargs]For keyword parameters, you can use ** to pack them into a dictionary. For example, >>> def my_print_kwargs[msg, **kwargs]:
print[msg]
for key, value in kwargs.items[]:
print['{}: {}'.format[key, value]]
>>> my_print_kwargs['hello', name='Peter', age=24]
hello
name: Peter
age: 24Unpacking Dictionary into Keyword Arguments [**dict]Similarly, you can also use **
to unpack a dictionary into individual keyword arguments >>> def my_print_kwargs[msg, **kwargs]:
print[msg]
for key, value in kwargs.items[]:
print['{}: {}'.format[key, value]]
>>> dict = {'k1':'v1', 'k2':'v2', 'k3':'v3'}
>>> my_print_kwargs['hello', **dict]
hello
k1: v1
k2: v2
k3: v3Using both *args and **kwargsYou can use both *args and **kwargs in your function definition. Place *args before **kwargs. For example,
>>> def my_print_all_args[*args, **kwargs]:
for item in args:
print[item]
for key, value in kwargs.items[]:
print['%s: %s' % [key, value]]
>>> my_print_all_args['a', 'b', 'c', name='Peter', age=24]
a
b
c
name: Peter
age: 24
>>> lst = [1, 2, 3]
>>> dict = {'name': 'peter'}
>>> my_print_all_args[*lst, **dict]
1
2
3
name: peterFunction OverloadingPython does NOT support Function Overloading like Java/C++ [where the same function name can have different versions differentiated by their parameters]. Function Return ValuesYou can return
multiple values from a Python function, e.g., >>> def my_fun[]:
return 1, 'a', 'hello'
>>> x, y, z = my_fun[]
>>> z
'hello'
>>> my_fun[]
[1, 'a', 'hello']It seems that Python function can return multiple values. In fact, a tuple that packs all the return values is returned. Recall that a tuple is actually formed through the commas, not the parentheses, e.g., >>> x = 1, 'a'
>>> x
[1, 'a']Types Hints via Function AnnotationsFrom Python 3.5, you can provide type hints via function annotations in the form of: def say_hello[name:str] -> str:
return 'hello, ' + name
say_hello['Peter']The type hints annotations are usually ignored, and merely serves as
documentation. But there are external library that can perform the type check. Read: "PEP 484 -- Type Hints". Modules, Import-Statement and PackagesModulesA Python module is a file containing Python codes - including statements, variables, functions and classes. It shall be saved with file extension of ".py". The module name is the filename, i.e., a module shall be saved
as ".py". By convention, modules names shall be short and all-lowercase [optionally joined with underscores if it improves readability]. A module typically begins with a triple-double-quoted documentation string [doc-string] [available in .__doc__], followed by variable, function and class definitions. Example: The greet ModuleCreate a module called greet and save as "greet.py" as follows: msg = 'Hello'
def greet[name]:
print['{}, {}'.format[msg, name]]This greet module defines a variable
msg and a function greet[]. The import statementTo use an external module in your script, use the import statement: import
import , , ...
import as Once imported, you can reference the module's attributes as .. You can use the import-as to assign a new module name to avoid module name conflict. For example, to use the greet module created earlier: $ cd /path/to/target-module
$ python3
>>> import greet
>>> greet.greet['Peter']
Hello, Peter
>>> print[greet.msg]
Hello
>>> greet.__doc__
'greet.py: the greet module with attributes msg and greet[]'
>>> greet.__name__
'greet'
>>> dir[greet]
['__built-ins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__',
'__package__', '__spec__', 'greet', 'msg']
>>> help[greet]
Help on module greet:
NAME
greet
DESCRIPTION
...doc-string...
FUNCTIONS
greet[name]
DATA
msg = 'Hello'
FILE
/path/to/greet.py
>>> import greet as grt
>>> grt.greet['Paul']
Hello, PaulThe import statements should be grouped in this order: - Python's standard library
- Third party libraries
- Local application libraries
The from-import StatementThe syntax is: from import
from import , , ...
from import *
from import as With the from-import statement, you can reference the imported attributes using directly, without qualifying with the . For example, >>> from greet import greet, msg as message
>>> greet['Peter']
Hello, Peter
>>> message
'Hello'
>>> msg
NameError: name 'msg' is not definedimport vs. from-importThe from-import statement actually loads the entire module [like import statement]; and NOT just the imported attributes. But it
exposes ONLY the imported attributes to the namespace. Furthermore, you can reference them directly without qualifying with the module name. For example, let create the following module called imtest.py for testing import vs. from-import: x = 1
y = 2
print['x is: {}'.format[x]]
def foo[]:
print['y is: {}'.format[y]]
def bar[]:
foo[]Let's try out import: $ python3
>>> import imtest
x is: 1
>>> imtest.y
2
>>> imtest.bar[]
y is: 2Now, try the from-import and note that the entire module is loaded, just like the import statement. $ python3
>>> from imtest import x, bar
x is: 1
>>> x
1
>>> bar[]
y is: 2
>>> foo[]
NameError: name 'foo' is not definedConditional ImportPython supports conditional import too. For example, if ....:
import xxx
else:
import yyysys.path
and PYTHONPATH/PATH environment variablesThe environment variable PATH shall include the path to Python Interpreter "python3". The Python module search path is maintained in a Python variable path of the sys module, i.e. sys.path. The sys.path is initialized from the environment variable PYTHONPATH, plus an installation-dependent default. The environment variable PYTHONPATH is empty by default. For example, >>> import sys
>>> sys.path
['', '/usr/lib/python3.5', '/usr/local/lib/python3.5/dist-packages',
'/usr/lib/python3.5/dist-packages', ...]sys.path default
includes the current working directory [denoted by an empty string], the standard Python directories, plus the extension directories in dist-packages. The imported modules must be available in one of the sys.path entries. >>> import some_mod
ImportError: No module named 'some_mod'
>>> some_mod.var
NameError: name 'some_mod' is not definedTo show the PATH and PYTHONPATH environment variables, use one of these commands: > echo %PATH%
> set PATH
> PATH
> echo %PYTHONPATH%
> set PYTHONPATH
$ echo $PATH
$ printenv PATH
$ echo $PYTHONPATH
$ printenv PYTHONPATHReloading Module using imp.reload[] or importlib.reload[]If you modify a module, you can use reload[] function of the imp [for
import] module to reload the module, for example, >>> import greet
>>> import imp
>>> imp.reload[greet]NOTE: Since Python 3.4, the imp package is pending deprecation in favor of importlib. >>> import greet
>>> import importlib
>>> importlib.reload[greet]Template for Python Standalone ModuleThe following is a template of standalone module for performing a specific task: import
import
import
......
......
def main[]:
.......
if __name__ == '__main__':
main[]When you execute a Python module [via the Python Interpreter], the __name__ is set to '__main__'. On the other hand, when a module is imported, its __name__ is set to the module name. Hence, the
above module will be executed if it is loaded by the Python interpreter, but not imported by another module. Example: [TODO] PackagesA module contains attributes [such as variables, functions and classes]. Relevant modules [kept in the same directory] can be grouped into a package. Python also supports sub-packages [in sub-directories]. Packages and sub-packages are a way of organizing Python's module namespace by using "dotted names" notation, in the form of '....'.
To create a Python package: - Create a directory and named it your package's name.
- Put your modules in it.
- Create a '__init__.py' file in the directory.
The '__init__.py' marks the directory as a package. For example, suppose that you have this directory/file structure: myapp/
|
+ mypack1/
| |
| + __init__.py
| + mymod1_1.py
| + mymod1_2.py
|
+ mypack2/
|
+ __init__.py
+ mymod2_1.py
+ mymod2_2.py If 'myapp' is in your 'sys.path', you can import 'mymod1_1' as: import mypack1.mymod1_1
from mypack1 import mymod1_1 Without the '__init__.py', Python will NOT search the 'mypack1' directory for 'mymod1_1'. Moreover, you cannot reference
modules in the 'mypack1' directory directly [e.g., 'import mymod1_1'] as it is not in the 'sys.path'. Attributes in '__init__.py'The '__init__.py' file is usually empty, but it can be used to initialize the package such as exporting selected portions of the package under more convenient name, hold convenience functions, etc. The attributes of the '__init__.py' module can be accessed via the package name directly [i.e., '.' instead of '..']. For example, import mypack1
from mypack1 import myattr1
Sub-PackagesA package can contain sub-packages too. For example, myapp/
|
+ mypack1/
|
+ __init__.py
+ mymod1_1.py
|
+ mysubpack1_1/
| |
| + __init__.py
| + mymod1_1_1.py
| + mymod1_1_2.py
|
+ mysubpack1_2/
|
+ __init__.py
+ mymod1_2_1.py Clearly, the package's dot structure corresponds to the directory structure. Relative from-importIn the from-import statement, you can use . to refer to the current package and .. to refer to the parent package. For example, inside 'mymod1_1_1.py', you can write: from . import mymod1_1_2
from .. import mymod1_1
from .mymod1_1_2 import attr
from ..mysubpack1_2 import mymod1_2_1Take note that in Python, you write '.mymod1_1_2', '..mysubpack1_2' by omitting the separating dot [instead of
'..mymod1_1_2', '...mysubpack1_2']. Circular Import Problem[TODO] Advanced Functions and NamespacesLocal Variables vs. Global VariablesNames created inside a function [i.e. within def statement] are local to the function and are available inside the function only. Names created outside all functions are global to that particular module [or file], but not available to the other modules. Global variables
are available inside all the functions defined in the module. Global-scope in Python is equivalent to module-scope or file-scope. There is NO all-module-scope in Python. For example, x = 'global'
def myfun[arg]:
y = 'local'
print[x]
print[y]
print[arg]
myfun['abc']
print[x]
#print[y] # locals are not visible outside the function
#print[arg]Function Variables [Variables of Function Object]In Python, a variable takes a value or object [such as int, str]. It can also take a function. For example, >>> def square[n]: return n * n
>>> square[5]
25
>>> sq = square
>>> sq[5]
25
>>> type[square]
>>> type[sq]
>>> square
>>> sq
A variable in Python can hold anything, a value, a function or an object. In Python,
you can also assign a specific invocation of a function to a variable. For example, >>> def square[n]: return n * n
>>> sq5 = square[5]
>>> sq5
25
>>> type[sq5]
Nested FunctionsPython supports nested functions, i.e., defining a function inside a function. For example, def outer[a]:
print['outer[] begins with arg =', a]
x = 1
def inner[b]:
print['inner[] begins with arg =', b]
y = 2
print['a = {}, x = {}, y = {}'.format[a, x, y]]
print['inner[] ends']
inner['bbb']
print['outer[] ends']
outer['aaa']The expected output is: outer begins with arg = aaa
inner begins with arg = bbb
a = aaa, x = 1, y = 2
inner ends
outer endsTake note that the inner function has read-access to all the attributes of the enclosing outer function, and the global variable of this module. Lambda Function [Anonymous Function]Lambda functions are anonymous
function or un-named function. They are used to inline a function definition, or to defer execution of certain codes. The syntax is: lambda arg1, arg2, ...: return_expressionFor example, >>> def f1[a, b, c]: return a + b + c
>>> f1[1, 2, 3]
6
>>> type[f1]
>>> f2 = lambda a, b, c: a + b + c
>>> f2[1, 2, 3]
6
>>> type[f2]
f1 and f2 do the same thing. Take note that return keyword is NOT needed inside the lambda function. Instead, it is similar to evaluating an expression to obtain a value. Lambda function, like ordinary function, can have default values for its parameters. >>> f3 = lambda a, b=2, c=3: a + b + c
>>> f3[1, 2, 3]
6
>>> f3[8]
13More usages for lambda function will be
shown later. Multiple Statements?Take note that the body of a lambda function is an one-liner return_expression. In other words, you cannot place multiple statements inside the body of a lambda function. You need to use a regular function for multiple statements. Functions are ObjectsIn Python, functions are objects [like instances of a class]. Like any object, - a function can be assigned to a variable;
- a function can be passed into a function as
an argument; and
- a function can be the return value of a function, i.e., a function can return a function.
Example: Passing a Function Object as a Function ArgumentA function name is a variable name that can be passed into another function as argument. def my_add[x, y]: return x + y
def my_sub[x, y]: return x - y
def my_apply[func, x, y]:
return func[x, y]
print[my_apply[my_add, 3, 2]]
print[my_apply[my_sub, 3, 2]]
print[my_apply[lambda x, y: x * y, 3, 2]] Example: Returning an Inner Function object from an Outer Functiondef my_outer[]:
def my_inner[]:
print['hello from inner']
return my_inner
result = my_outer[]
result[]
print[result] Example: Returning a Lambda Functiondef increase_by[n]:
return lambda x: x + n
plus_8 = increase_by[8]
print[plus_8[1]]
plus_88 = increase_by[88]
print[plus_88[1]]
print[increase_by[8][1]]
print[increase_by[88][1]]Function ClosureIn the above example,
n is not local to the lambda function. Instead, n is obtained from the outer function. When we assign increase_by[8] to plus_8, n takes on the value of 8 during the invocation. But we expect n to go out of scope after the outer function terminates. If this is the case, calling plus_8[1] would encounter an non-existent n? This problem is resolved via so called Function Closure. A closure is an inner function that is passed outside the
enclosing function, to be used elsewhere. In brief, the inner function creates a closure [enclosure] for its enclosing namespaces at definition time. Hence, in plus_8, an enclosure with n=8 is created; while in plus_88, an enclosure with n=88 is created. Take note that Python only allows the read access to the outer scope, but not write access. You can inspect the enclosure via function_name.func_closure, e.g., print[plus_8.func_closure]
print[plus_88.func_closure]
Functional Programming: Using Lambda Function in filter[],
map[], reduce[] and ComprehensionInstead of using a for-in loop to iterate through all the items in an iterable [sequence], you can use the following functions to apply an operation to all the items. This is known as functional programming or expression-oriented programming. Filter-map-reduce is popular in big data analysis [or data science]. - filter[func, iterable]: Return an iterator yielding those items of iterable for which func[item] is
True. For example, >>> lst = [11, 22, 33, 44, 55]
>>> filter[lambda x: x % 2 == 0, lst]
>>> list[filter[lambda x: x % 2 == 0, lst]]
[22, 44]
>>> for item in filter[lambda x: x % 2 == 0, lst]: print[item, end=' ']
22 44
>>> print[filter[lambda x: x % 2 == 0, lst]]
- map[func, iterable]: Apply [or Map or Transform] the function func on each item of the iterable. For example, >>> lst = [11, 22, 33, 44, 55]
>>> map[lambda x: x*x, lst]
>>> list[map[lambda x: x*x, lst]]
[121, 484, 1089, 1936, 3025]
>>> for item in map[lambda x: x*x, lst]: print[item, end=' ']
121 484 1089 1936 3025
>>> print[map[lambda x: x*x, lst]]
- reduce[func, iterable] [in module functools]: Apply the function of two arguments cumulatively to the items of a sequence, from left to right, so as to reduce the sequence to a single value, also known as aggregation. For example, >>> lst = [11, 22, 33, 44, 55]
>>> from functools import reduce
>>> reduce[lambda x,y: x+y, lst]
165
- filter-map-reduce: used frequently in big data analysis to obtain an aggregate value. >>> new_lst = list[map[lambda x: x*x, filter[lambda x: x % 2 == 0, lst]]]
>>> new_lst
[4, 36]
>>> from functools import reduce
>>> reduce[lambda x, y: x+y, map[lambda x: x*x, filter[lambda x: x % 2 == 0, lst]]]
40
- List comprehension: a one-liner to generate a list as discussed in the earlier section. e.g., >>> lst = [3, 2, 6, 5]
>>> new_lst = [x*x for x in lst if x % 2 == 0]
>>> new_lst
[4, 36]
>>> f = lambda x: x*x
>>> new_lst = [f[x] for x in lst if x % 2 == 0]
>>> new_lst
[4, 36]
>>> new_lst = [[lambda x: x*x][x] for x in lst if x % 2 == 0]
>>> new_lst
[4, 36]
These mechanism replace the traditional for-loop, and express their functionality in simple function calls. It is called functional programming, i.e., applying a series of functions [filter-map-reduce] over a collection. DecoratorsIn Python, a decorator is a callable [function] that takes a function as an argument and returns a replacement function.
Recall that functions are objects in Python, i.e., you can pass a function as argument, and a function can return an inner function. A decorator is a transformation of a function. It can be used to pre-process the function arguments before passing them into the actual function; or extending the behavior of functions that you don't want to modify, such as ascertain that the user has login and has the necessary permissions. Example: Decorating an 1-argument Functiondef clamp_range[func]:
def _wrapper[x]:
if x < 0:
x = 0
elif x > 100:
x = 100
return func[x]
return _wrapper
def square[x]: return x*x
print[clamp_range[square][5]]
print[clamp_range[square][111]] print[clamp_range[square][-5]]
square = clamp_range[square]
print[square[50]]
print[square[-1]]
print[square[101]] Notes:
- The decorator clamp_range[] takes a 1-argument function as its argument, and returns an replacement 1-argument function _wrapper[x], with its argument x clamped to [0,100], before applying the original function.
- In 'square=clamp_range[square]', we decorate the square[] function and assign the decorated [replacement] function to the same function name [confusing?!]. After the decoration, the square[] takes on a new decorated life!
Example: Using the @ symbolUsing
'square=clamp_range[square]' to decorate a function is messy?! Instead, Python uses the @ symbol to denote the replacement. For example, def clamp_range[func]:
def _wrapper[x]:
if x < 0:
x = 0
elif x > 100:
x = 100
return func[x]
return _wrapper
@clamp_range
def cube[x]: return x**3
print[cube[50]]
print[cube[-1]]
print[cube[101]] For Java programmers, do not confuse the Python decorator @ with Java's annotation like @Override. Example: Decorator with an Arbitrary Number of Function ArgumentsThe above example only work for one-argument function. You can use *args and/or **kwargs to handle variable number of arguments. For example,
the following decorator log all the arguments before the actual processing. def logger[func]:
def _wrapper[*args, **kwargs]:
print['The arguments are: {}, {}'.format[args, kwargs]]
return func[*args, **kwargs]
return _wrapper
@logger
def myfun[a, b, c=3, d=4]:
pass
myfun[1, 2, c=33, d=44]
myfun[1, 2, c=33] We can also modify our earlier clamp_range[] to handle an arbitrary number of arguments: def clamp_range[func]:
def _wrapper[*args]:
newargs = []
for item in args:
if item < 0:
newargs.append[0]
elif item > 100:
newargs.append[100]
else:
newargs.append[item]
return func[*newargs]
return _wrapper
@clamp_range
def my_add[x, y, z]: return x + y + z
print[my_add[1, 2, 3]]
print[my_add[-1, 5, 109]] The @wraps DecoratorDecorator can be hard to debug. This is because it wraps around and replaces the original function and hides variables like __name__ and __doc__. This can be solved by using the @wraps of functools, which modifies the signature of the replacement functions so they look
more like the decorated function. For example, from functools import wraps
def without_wraps[func]:
def _wrapper[*args, **kwargs]:
return func[*args, **kwargs]
return _wrapper
def with_wraps[func]:
@wraps[func]
def _wrapper[*args, **kwargs]:
return func[*args, **kwargs]
return _wrapper
@without_wraps
def fun_without_wraps[]:
pass
@with_wraps
def fun_with_wraps[]:
pass
print[fun_without_wraps.__name__]
print[fun_without_wraps.__doc__]
print[fun_with_wraps.__name__]
print[fun_with_wraps.__doc__] Example: Passing Arguments into DecoratorsLet's modify the earlier clamp_range decorator to take two arguments - min and max of the range. from functools import wraps
def clamp_range[min, max]:
def _decorator[func]:
@wraps[func]
def _wrapper[*args]:
newargs = []
for item in args:
if item < min:
newargs.append[min]
elif item > max:
newargs.append[max]
else:
newargs.append[item]
return func[*newargs]
return _wrapper
return _decorator
@clamp_range[1, 10]
def my_add[x, y, z]:
return x + y + z
print[my_add[1, 2, 3]]
print[my_add[-1, 5, 109]]
print[my_add.__name__]
print[my_add.__doc__] The decorator clamp_range takes the desired arguments and returns a wrapper function which takes a function argument [for the function to be decorated]. Confused?! Python has many fancy features that is not available in traditional languages like C/C++/Java! Namespace
Names, Namespaces and ScopeIn Python, a name is roughly analogous to a variable in other languages but with some extras. Because of the dynamic nature of Python, a name is applicable to almost everything, including variable, function, class/instance, module/package. Names defined inside a function are local. Names defined outside all functions are global for that module, and are accessible by all functions inside the module [i.e., module-global scope]. There is no
all-module-global scope in Python. A namespace is a collection of names [i.e., a space of names]. A scope refers to the portion of a program from where a names can be accessed without a qualifying prefix. For example, a local variable defined inside a function has local scope [i.e., it is available within the function, and NOT available outside the function]. Each Module has a Global NamespaceA module is a file containing attributes
[such as variables, functions and classes]. Each module has its own global namespace. Hence, you cannot define two functions or classes of the same name within a module. But you can define functions of the same name in different modules, as the namespaces are isolated. When you launch the interactive shell, Python creates a module called __main__, with its associated global namespace. All subsequent names are added into __main__'s namespace. When you import a module via 'import '
under the interactive shell, only the is added into __main__'s namespace. You need to access the names [attributes] inside via .. In other words, the imported module retains its own namespace and must be prefixed with . inside __main__. [Recall that the scope of a name is the portion of codes that can access it without prefix.] However, if you import an attribute via 'from import ' under the interactive shell, the is added into __main__'s namespace, and
you can access the directly without prefixing with the . On the other hand, when you import a module inside another module [instead of interactive shell], the imported is added into the target module's namespace [instead of __main__ for the interactive shell]. The built-in functions are kept in a module called __built-in__, which is imported into __main__ automatically. The globals[], locals[] and dir[] Built-in
FunctionsYou can list the names of the current scope via these built-in functions: - globals[]: return a dictionary [name-value pairs] containing the current scope's global variables.
- locals[]: return a dictionary [name-value pairs] containing the current scope's local variables. If locals[] is issued in global scope, it returns the same outputs as globals[].
- dir[]: return a list of local names in the current scope, which is equivalent to locals[].keys[].
- dir[obj]:
return a list of the local names for the given object.
For example,
$ python3
>>> globals[]
{'__name__': '__main__',
'__built-ins__': ,
'__doc__': None,
'__package__': None,
'__spec__': None,
'__loader__': }
>>> __name__
'__main__'
>>> locals[]
...same outputs as global[] under the global-scope...
>>> dir[]
['__built-ins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__']
>>> x = 88
>>> globals[]
{'x': 88, ...}
>>> dir[]
['__built-ins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'x']
>>> import random
>>> globals[]
{'x': 88,
'random': ,
......}
>>> from math import pi
>>> globals[]
{'x': 88,
'pi': 3.141592653589793,
'random': ,
......}To show the difference between locals and globals, we need to define a function to create a local scope. For example, $ python3
>>> x = 88
>>> def myfun[arg]:
y = 99
print[x]
print[globals[]]
print[locals[]]
print[dir[]]
>>> myfun[11]
88
{'__built-ins__': ,
'myfun': ,
'__name__': '__main__',
'__package__': None,
'__spec__': None,
'__doc__': None,
'__loader__': ,
'x': 88}
{'y': 99, 'arg': 11}
['arg', 'y'] More on Module's Global NamespaceLet's create two modules: mod1 and mod2, where mod1 imports mod2, as follows: import mod2
mod1_var = 'mod1 global variable'
print['Inside mod1, __name__ = ', __name__]
if __name__ == '__main__':
print['Run module 1']mod2_var = 'mod2 global variable'
print['Inside mod2, __name__ = ', __name__]
if __name__ == '__main__':
print['Run module 2']Let's import mod1 [which in turn import mod2] under the interpreter shell, and check the namespaces: >>> import mod1
Inside mod2, __name__ = mod2
Inside mod1, __name__ = mod1
>>> dir[]
['__built-ins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'mod1']
>>> dir[mod1]
['__built-ins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'mod1_var', 'mod2']
>>> dir[mod2]
NameError: name 'mod2' is not defined
>>> dir[mod1.mod2]
['__built-ins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'mod2_var']
Take note that the interpreter's current scope __name__ is __main__. It's namespace contains mod1 [imported]. The mod1's namespace contains mod2 [imported] and mod1_var. To refer to mod2, you need to go thru mod1, in the form of mod1.mod2. The mod1.mod2's namespace contains mod2_var. Now, let run mod1 instead, under IDLE3, and check the namespaces: Inside mod2, __name__ = mod2
Inside mod1, __name__ = __main__
Run module 1
>>> dir[]
['__built-ins__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'mod1_var', 'mod2']
>>> dir[mod2]
['__built-ins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'mod2_var']Take note that the current scope's name is again __main__, which is the executing module mod1.
Its namespace contains mod2 [imported] and mod1_var. Name ResolutionWhen you ask for a name [variable], says x, Python searches the LEGB namespaces, in this order, of the current scope: - L: Local namespace which is specific to the current function
- E: for nested function, the Enclosing function's namespace
- G: Global namespace for the current module
- B: Built-in namespace for all the modules
If x cannot be found, Python
raises a NameError. Modifying Global Variables inside a FunctionRecall that names created inside a function are local, while names created outside all functions are global for that module. You can "read" the global variables inside all functions defined in that module. For example, x = 'global'
def myfun[]:
y = 'local'
print[y]
print[x]
myfun[]
print[x]
#print[y] If you assign a value to a name inside a function, a local name is created, which hides the global name. For example, x = 'global'
def myfun[]:
x = 'change'
print[x]
myfun[]
print[x] To modify a global variable inside a function, you need to
use a global statement to declare the name global; otherwise, the modification [assignment] will create a local variable [see above]. For example, x = 'global'
def myfun[]:
global x
x = 'change'
print[x]
myfun[]
print[x] For nested functions, you need to use the nonlocal statement in the inner function to modify names in the enclosing outer function. For example, def outer[]:
count = 0
def inner[]:
nonlocal count
count += 1
print[count]
inner[]
print[count]
outer[]To modify a global variable inside a nested function, declare it via global statement too. For example, count = 100
def outer[]:
count = 0
def inner[]:
global count
count += 1
print[count]
inner[]
print[count]
outer[]
print[count] In summary, - The order for name resolution [for
names inside a function] is: local, enclosing function for nested def, global, and then the built-in namespaces [i.e., LEGB].
- However, if you assign a new value to a name, a local name is created, which hides the global name.
- You need to declare via global statement to modify globals inside the function. Similarly, you need to declare via nonlocal statement to modify enclosing local names inside the nested function.
More on global Statement
The global statement is necessary if you are changing the reference to an object [e.g. with an assignment]. It is not needed if you are just mutating or modifying the object. For example, >>> a = []
>>> def myfun[]:
a.append['hello']
>>> myfun[]
>>> a
['hello']In the above example, we modify the contents of the array. The global statement is not needed. >>> a = 1
>>> def myfun[]:
global a
a = 8
>>> myfun[]
>>> a
8In the above example, we are modifying the reference to the variable. global is needed, otherwise, a local variable will be created inside the function. Built-in Namespace
The built-in namespace is defined in the __built-ins__ module, which contains built-in functions such as len[], min[], max[], int[], float[], str[], list[], tuple[] and etc. You can use help[__built-ins__] or dir[__built-ins__] to list the attributes of the __built-ins__ module. [TODO] del StatementYou can use del statement to remove names from the namespace, for example, >>> del x, pi
>>> globals[]
...... x and pi removed ......
>>> del random
>>> globals[]
...... random module removed ......If you override a built-in function, you could also use del to remove it from
the namespace to recover the function from the built-in space. >>> len = 8
>>> len['abc']
TypeError: 'int' object is not callable
>>> del len
>>> len['abc']
3Assertion and Exception Handlingassert StatementYou can use assert statement to test a certain assertion [or constraint]. For example, if x is supposed to be 0 in a certain part of the program, you can use the assert statement to test this constraint. An AssertionError will be raised if x is not zero. For example, >>> x = 0
>>> assert x == 0, 'x is not zero?!'
>>> x = 1
>>> assert x == 0, 'x is not zero?!'
......
AssertionError: x is not zero?!The assertions
are always executed in Python. SyntaxThe syntax for assert is: assert test, error-messageIf the test if True, nothing happens; otherwise, an AssertionError will be raised with the error-message. ExceptionsIn Python, errors detected during execution are called exceptions. For example, >>> 1/0
ZeroDivisionError: division by zero
>>> zzz
NameError: name 'zzz' is not defined
>>> '1' + 1
TypeError: Can't convert 'int' object to str implicitly
>>> lst = [0, 1, 2]
>>> lst[3]
IndexError: list index out of range
>>> lst.index[8]
ValueError: 8 is not in list
>>> int['abc']
ValueError: invalid literal for int[] with base 10: 'abc'
>>> tup = [1, 2, 3]
>>> tup[0] = 11
TypeError: 'tuple' object does not support item assignmentWhenever an exception is raised, the program terminates abruptly. try-except-else-finallyYou can use try-except-else-finally exception handling facility to prevent
the program from terminating abruptly. Example 1: Handling Index out-of-range for List Accessdef get_item[seq, index]:
try:
result = seq[index]
print['try succeed']
except IndexError:
result = 0
print['Index out of range']
except:
result = 0
print['other exception']
else:
print['no exception raised']
finally:
print['run finally']
print['continue after try-except']
return result
print[get_item[[0, 1, 2, 3], 1]]
print['-----------']
print[get_item[[0, 1, 2, 3], 4]] The expected outputs are: try succeed
no exception raised
run finally
continue after try-except
1
-----------
Index out of range
run finally
continue after try-except
0The exception handling process for try-except-else-finally is: - Python runs the statements in the try-block.
- If no exception is raised in all the statements of the try-block, all the except-blocks are skipped, and the program continues to the next statement after the try-except statement.
- However, if an exception
is raised in one of the statement in the try-block, the rest of try-block will be skipped. The exception is matched with the except-blocks. The first matched except-block will be executed. The program then continues to the next statement after the try-except statement, instead of terminates abruptly. Nevertheless, if none of the except-blocks is matched, the program terminates abruptly.
- The else-block will be executable if no exception is raised.
- The
finally-block is always executed for doing house-keeping tasks such as closing the file and releasing the resources, regardless of whether an exception has been raised.
SyntaxThe syntax for try-except-else-finally is: try:
statements
except exception_1:
statements
except [exception_2, exception_3]:
statements
except exception_4 as var_name:
statements
except:
statements
else:
statements
finally:
statements The try-block [mandatory] must follow by at least one except or finally block. The rests are optional. CAUTION: Python 2 uses older syntax of "except exception-4, var_name:", which should be re-written as "except exception-4 as var_name:" for portability. Example 2: Input
Validation>>> while True:
try:
x = int[input['Enter an integer: ']]
break
except ValueError:
print['Invalid input! Try again...']
Enter an integer: abc
Wrong input! Try again...
Enter an integer: 11.22
Wrong input! Try again...
Enter an integer: 123raise StatementYou can manually raise an exception via the raise statement, for example, >>> raise IndexError['out-of-range']
IndexError: out-of-rangeThe syntax is: raise exception_class_name
raise exception_instance_name
raise A raise without argument in the except block re-raise the exception to the outer block, e.g., try:
......
except:
raise Built-in Exceptions- BaseException, Exception, StandardError: base classes
- ArithmeticError: for OverflowError, ZeroDivisionError, FloatingPointError.
- BufferError:
- LookupError: for IndexError, KeyError.
- Environment:
for IOError, OSError.
- [TODO] more
User-defined ExceptionYou can defined your own exception by sub-classing the Exception class. Exampleclass MyCustomError[Exception]:
def __init__[self, value]:
self.value = value
def __str__[self]:
return repr[self.value]
try:
raise MyCustomError['an error occurs']
print['after exception']
except MyCustomError as e:
print['MyCustomError: ', e.value]
else:
print['running the else block']
finally:
print['always run the finally block']with-as Statement and Context ManagersThe syntax of the with-as statement is as follows: with ... as ...:
statements
with ... as ..., ... as ..., ...:
statementsPython’s with statement supports the concept of a runtime context defined by a context manager. In programming, context can be seen as a bucket to pass information around, i.e., the
state at a point in time. Context Managers are a way of allocating and releasing resources in the context. Example 1with open['test.log', 'r'] as infile:
for line in infile:
print[line]This is equivalent to: infile = open['test.log', 'r']
try:
for line in infile:
print[line]
finally:
infile.close[]The with-statement's context manager acquires, uses, and releases the context [of the file] cleanly, and eliminate a bit of boilerplate. However, the with-as statement is applicable to certain objects only, such as file; while try-finally can be applied to all. Example 2:with open['in.txt', 'r'] as infile, open['out.txt', 'w'] as outfile:
for line in infile:
outfile.write[line]Frequently-Used
Python Standard Library ModulesPython provides a set of standard library. [Many non-standard libraries are provided by third party!] To use a module, use 'import ' or 'from import ' to import the entire module or a selected attribute. You can use 'dir[]' to list all the attributes of the module, 'help[]' or 'help[]' to read the documentation page. For example, >>> import math
>>> dir[math]
['e', 'pi', 'sin', 'cos', 'tan', 'tan2', ....]
>>> help[math]
......
>>> help[math.atan2]
......
>>> math.atan2[3, 0]
1.5707963267948966
>>> math.sin[math.pi / 2]
1.0
>>> math.cos[math.pi / 2]
6.123233995736766e-17
>>> from math import pi
>>> pi
3.141592653589793math and cmath ModulesThe math module provides access to the mathematical
functions defined by the C language standard. The commonly-used attributes are: - Constants: pi, e.
- Power and exponent: pow[x,y], sqrt[x], exp[x], log[x], log2[x], log10[x]
- Converting float to int: ceil[x], floor[x], trunc[x].
- float operations: fabs[x], fmod[x]
- hypot[x,y] [=sqrt[x*x + y*y]]
- Conversion between degrees and radians: degrees[x], radians[x].
- Trigonometric functions: sin[x], cos[x], tan[x], acos[x],
asin[x], atan[x], atan2[x,y].
- Hyperbolic functions: sinh[x], cosh[x], tanh[x], asinh[x], acosh[x], atanh[x].
For examples, >>> import math
>>> dir[math]
......
>>> help[math]
......
>>> help[math.trunc]
......
>>> x = 1.5
>>> type[x]
>>> math.floor[x]
1
>>> type[math.floor[x]]
>>> math.ceil[x]
2
>>> math.trunc[x]
1
>>> math.floor[-1.5]
-2
>>> math.ceil[-1.5]
-1
>>> math.trunc[-1.5]
-1
In addition, the cmath module provides mathematical functions for complex numbers. See Python documentation for details. statistics ModuleThe statistics module computes the basic statistical properties such as mean, median, variance, and etc. [Many third-party vendors
provide advanced statistics packages!] For examples, >>> import statistics
>>> dir[statistics]
['mean', 'median', 'median_grouped', 'median_high', 'median_low', 'mode', 'pstdev', 'pvariance', 'stdev', 'variance', ...]
>>> help[statistics]
......
>>> help[statistics.pstdev]
......
>>> data = [5, 7, 8, 3, 5, 6, 1, 3]
>>> statistics.mean[data]
4.75
>>> statistics.median[data]
5.0
>>> statistics.stdev[data]
2.3145502494313788
>>> statistics.variance[data]
5.357142857142857
>>> statistics.mode[data]
statistics.StatisticsError: no unique mode; found 2 equally common valuesrandom ModuleThe module random can be used to generate various pseudo-random numbers. For examples, >>> import random
>>> dir[random]
......
>>> help[random]
......
>>> help[random.random]
......
>>> random.random[]
0.7259532743815786
>>> random.random[]
0.9282534690123855
>>> random.randint[1, 6]
3
>>> random.randrange[6]
0
>>> random.choice[['apple', 'orange', 'banana']]
'apple'sys ModuleThe module sys [for system] provides system-specific parameters and functions. The commonly-used are: - sys.exit[[exit_status=0]]: exit the program by raising the SystemExit exception. If used inside a try, the finally clause is
honored. The optional argument exit_status can be an integer [default to 0 for normal termination, or non-zero for abnormal termination]; or any object [e.g., sys.exit['an error message']].
- sys.path: A list of module search-paths. Initialized from the environment variable PYTHONPATH, plus installation-dependent default entries. See earlier example.
- sys.stdin, sys.stdout, sys.stderr: standard input, output and error stream.
- sys.argv: A list of
command-line arguments passed into the Python script. argv[0] is the script name. See example below.
Example: Command-Line ArgumentsThe command-line arguments are kept in sys.argv as a list. For example, create the following script called "test_argv.py": import sys
print[sys.argv]
print[len[sys.argv]] Run the script: $ python3 test_argv.py
['test_argv.py']
1
$ python3 test_argv.py hello 1 2 3 apple orange
['test_argv.py', 'hello', '1', '2', '3', 'apple', 'orange']
7logging ModuleThe logging moduleThe logging module supports a flexible event logging system for your applications and
libraries. The logging supports five levels: - logging.DEBUG: Detailed information meant for debugging.
- logging.INFO: Confirmation that an event takes place as expected.
- logging.WARNING: Something unexpected happened, but the application is still working.
- logging.ERROR: The application does not work as expected.
- logging.CRITICAL: Serious error, the application may not be able to continue.
The logging functions are: - logging.basicConfig[**kwargs]: Perform basic configuration
of the logging system. The keyword arguments are: filename, filemode [default to append 'a'], level [log this level and above], and etc.
- logging.debug[msg, *args, **kwargs], logging.info[], logging.warning[], logging.error[], logging.critical[]: Log the msg at the specific level. The args are merged into msg using formatting specifier.
- logging.log[level, msg, *args, **kwargs]: General logging function, at the given log level.
Basic Logging via logging.basicConfig[]For example, import logging
logging.basicConfig[filename='myapp.log', level=logging.DEBUG]
logging.debug['A debug message']
logging.info['An info message {}, {}'.format['apple', 'orange']]
logging.error['error {}, some error messages'.format[1234]]The logging functions
support printf-like format specifiers such as %s, %d, with values as function arguments [instead of via % operator in Python]. Run the script. A log file myapp.log would be created, with these records: DEBUG:root:A debug message
INFO:root:An info message apple, orange
ERROR:root:error 1234, some error messagesBy default, the log records include the log-level and logger-name [default of root] before the message. Getting the Log Level from a Configuration FileLog levels, such as logging.DEBUG and logging.INFO, are stored as certain integers in the logging
module. For example, >>> import logging
>>> logging.DEBUG
10
>>> logging.INFO
20The log level is typically read from a configuration file, in the form of a descriptive string. The following example shows how to convert a string log-level [e.g., 'debug'] to the numeric log-level [e.g., 10] used by logging module: import logging
str_level = 'info'
numeric_level = getattr[logging, str_level.upper[], None]
if not isinstance[numeric_level, int]:
raise ValueError['Invalid log level: {}'.format[str_level]]
logging.basicConfig[level=numeric_level]
logging.debug['a debug message']
logging.info['an info message']
logging.error['an error message'] Log Record FormatTo set the log message format, use the format keyword: import logging
logging.basicConfig[
format='%[asctime]s|%[levelname]s|%[name]s|%[pathname]s:%[lineno]d|%[message]s',
level=logging.DEBUG]where asctime for date/time, levelname for log level, name for logger name, pathname for full-path filename [filename
for filename only], lineno [int] for the line number, and message for the log message. Advanced Logging: Logger, Handler, Filter and FormatterSo far, we presented the basic logging facilities. The logging library is extensive and organized into these components: - Loggers: expose the methods to application for logging.
- Handlers: send the log records created by the loggers to the appropriate destination, such as file, console [sys.stderr], email via SMTP, or
network via HTTP/FTP.
- Filters: decide which log records to output.
- Formatters: specify the layout format of log records.
LoggersTo create a Logger instance, invoke the logging.getLogger[logger-name], where the optional logger-name specifies the logger name [default of root]. The Logger's methods falls into two categories: configuration and logging. The commonly-used logging methods are: debug[], info[], warning[], error[], critical[] and the general log[].
The commonly-used configuration methods are: - setLevel[]
- addHandler[] and removeHandler[]
- addFilter[] and removeFilter[]
HandlersThe logging library provides handlers like StreamHandler [sys.stderr, sys.stdout], FileHandler, RotatingFileHandler, and SMTPHandler [emails]. The commonly-used methods are: - setLevel[]: The logger's setLevel[] determines which message levels to be passed to the handler; while the handler's setLevel[] determines which message level to be sent to the
destination.
- setFormatter[]: for formatting the message sent to the destination.
- addFilter[] and removeFilter[]
You can add more than one handlers to a logger, possibly handling different log levels. For example, you can add a SMTPHandler to receive emails for ERROR level; and a RotatingFileHandler for INFO level. FormattersAttach to a handler [via .setFormatter[]] to format the log messages. Example: Using Logger with Console Handler and a Formatterimport logging
logger = logging.getLogger['MyApp']
logger.setLevel[logging.INFO]
ch = logging.StreamHandler[]
ch.setLevel[logging.INFO]
formatter = logging.Formatter['%[asctime]s|%[name]s|%[levelname]s|%[message]s']
ch.setFormatter[formatter]
logger.addHandler[ch]
logger.debug['a debug message']
logger.info['an info message']
logger.warn['a warn message']
logger.error['error %d, an error message', 1234]
logger.critical['a critical message']- There
is probably no standard for log record format [unless you have an analysis tool in mind]?! But I recommend that you choose a field delimiter which does not appear in the log messages, for ease of processing of log records [e.g., export to spreadsheet].
The expected outputs are: 2015-12-09 00:32:33,521|MyApp|INFO|an info message
2015-12-09 00:32:33,521|MyApp|WARNING|a warn message
2015-12-09 00:32:33,521|MyApp|ERROR|error 1234: an error message
2015-12-09 00:32:33,521|MyApp|CRITICAL|a critical messageExample: Using Rotating Log Files with RotatingFileHandlerimport logging
from logging.handlers import RotatingFileHandler
config = {
'loggername' : 'myapp',
'logLevel' : logging.INFO,
'logFilename' : 'test.log',
'logFileBytes': 300,
'logFileCount': 3}
logger = logging.getLogger[config['loggername']]
logger.setLevel[config['logLevel']]
handler = RotatingFileHandler[
config['logFilename'],
maxBytes=config['logFileBytes'],
backupCount=config['logFileCount']]
handler.setLevel[config['logLevel']]
handler.setFormatter[logging.Formatter[
"%[asctime]s|%[levelname]s|%[message]s|%[filename]s:%[lineno]d"]]
logger.addHandler[handler]
logger.info['An info message']
logger.debug['A debug message']
for i in range[1, 10]:
logger.error['Error message %d', i]- We keep all the logging parameters in a dictionary, which are usually retrieved from a configuration
file.
- In the constructor of RotatingFileHandler, the maxBytes sets the log file size-limit; the backupCount appends '.1', '.2', etc to the old log files, such that '.1' is always the newer backup of the log file. Both maxBytes and backupCount default to 0. If either one is zero, roll-over never occurs.
- The above example produces 4 log files: test.log, test.log.1 to test.log.3. The file being written to is always test.log. When this file is filled, it is renamed to test.log.1; and if
test.log.1 and test.log.2 exist, they will be renamed to test.log.2 and test.log.3 respectively, with the old test.log.3 deleted.
Example: Using an Email Log for CRITICAL Level and Rotating Log Files for INFO Levelimport logging
from logging.handlers import RotatingFileHandler, SMTPHandler
config = {
'loggername' : 'myapp',
'fileLogLevel' : logging.INFO,
'logFilename' : 'test.log',
'logFileBytes' : 300,
'logFileCount' : 5,
'emailLogLevel': logging.CRITICAL,
'smtpServer' : 'your_smtp_server',
'email' : '',
'emailAdmin' : ''}
logger = logging.getLogger[config['loggername']]
logger.setLevel[config['fileLogLevel']]
fileHandler = RotatingFileHandler[
config['logFilename'],
maxBytes=config['logFileBytes'],
backupCount=config['logFileCount']]
fileHandler.setLevel[config['fileLogLevel']]
fileHandler.setFormatter[logging.Formatter[
"%[asctime]s|%[levelname]s|%[message]s|%[filename]s:%[lineno]d"]]
emailHandler = SMTPHandler[
config['smtpServer'],
config['email'],
config['emailAdmin'],
'%s - CRITICAL ERROR' % config['loggername']]
emailHandler.setLevel[config['emailLogLevel']]
logger.addHandler[fileHandler]
logger.addHandler[emailHandler]
logger.debug['A debug message']
logger.info['An info message']
logger.warning['A warning message']
logger.error['An error message']
logger.critical['A critical message']Example: Separating ERROR Log and INFO Log with Different Formatimport logging, sys
from logging.handlers import RotatingFileHandler
class MaxLevelFilter[logging.Filter]:
def __init__[self, maxlevel]:
self.maxlevel = maxlevel
def filter[self, record]:
return [record.levelno >> import datetime
>>> dir[datetime]
['MAXYEAR', 'MINYEAR', 'date', 'datetime', 'datetime_CAPI', 'time', 'timedelta', 'timezone', 'tzinfo', ...]
>>> dir[datetime.date]
['today', ...]
>>> from datetime import date
>>> today = date.today[]
>>> today
datetime.date[2016, 6, 17]
>>> aday = date[2016, 5, 1]
>>> aday
datetime.date[2016, 5, 1]
>>> diff = today - aday
>>> diff
datetime.timedelta[47]
>>> dir[datetime.timedelta]
['days', 'max', 'microseconds', 'min', 'resolution', 'seconds', 'total_seconds', ...]
>>> diff.days
47smtplib and email ModulesThe SMTP [Simple Mail Transfer Protocol] is a protocol, which handles
sending email and routing email between mail servers. Python provides a smtplib module, which defines an SMTP client session object that can be used to send email to any Internet machine with an SMTP listener daemon. To use smtplib: import smtplib
smtpobj = smtplib.SMTP[[host [,port [, local_hostname [, timeout]]]]]
......
smtpobj.sendmail[form_addr, to_addrs, msg]
smtpobj.quit[]
The email module can be used to construct an email message. [TODO] more json ModuleJSON [JavaScript Object Notation] is a lightweight data interchange format inspired by JavaScript object literal syntax. The
json module provides implementation for JSON encoder and decoder. - json.dumps[python_obj]: Serialize python_obj to a JSON-encoded string ['s' for string].
- json.loads[json_str]: Create a Python object from the given JSON-encoded string.
- json.dump[python_obj, file_obj]: Serialize python_obj to the file.
- json.load[file_obj]: Create a Python object by reading the given file.
For example, >>> import json
>>> lst = [123, 4.5, 'hello', True]
>>> json_lst = json.dumps[lst]
>>> json_lst
'[123, 4.5, "hello", true]'
>>> dct = {'a': 11, 2: 'b', 'c': 'cc'}
>>> json_dct = json.dumps[dct]
>>> json_dct
'{"a": 11, "c": "cc", "2": "b"}'
>>> lst_decoded = json.loads[json_lst]
>>> lst_decoded
[123, 4.5, 'hello', True]
>>> dct_decoded = json.loads[json_dct]
>>> dct_decoded
{'a': 11, 'c': 'cc', '2': 'b'}
>>> f = open['json.txt', 'w']
>>> json.dump[dct, f]
>>> f.close[]
>>> f = open['json.txt', 'r']
>>> dct_decoded_from_file = json.load[f]
>>> dct_decoded_from_file
{'a': 11, 'c': 'cc', '2': 'b'}
>>> f.seek[0]
0
>>> f.read[]
'{"a": 11, "c": "cc", "2": "b"}'
>>> f.close[]pickle and cPickle ModulesThe json module [described earlier] handles lists
and dictionaries, but serializing arbitrary class instances requires a bit of extra effort. On the other hand, the pickle module implements serialization and de-serialization of any Python object. Pickle is a protocol which allows the serialization of arbitrarily complex Python objects. It is specific to the Python languages and not applicable to other languages. The pickle module provides the same functions as the json module: - pickle.dumps[python_obj]: Return the
pickled representation of the python_obj as a string.
- pickle.loads[pickled_str]: Construct a Python object from pickled_str.
- pickle.dump[python_obj, file_obj]: Write a pickled representation of the python_obj to file_obj.
- pickle.load[file_obj]: Construct a Python object reading from the file_obj.
The module cPickle is an improved version of pickle. signal moduleSignals [software interrupt] are a limited form of asynchronous inter-process
communication, analogous to hardware interrupts. It is generally used by the operating system to notify processes about certain issues/states/errors, like division by zero, etc. The signal module provides mechanisms to use signal handlers in Python. signal.signal[]The signal.signal[] method takes two arguments: the signal number to handle, and the handling function. For example, import sys, signal, time
def my_signal_handler[signalnum, handler]:
print['Signal received %d: %s' % [signalnum, handler]];
signal.signal[signal.SIGINT, my_signal_handler];
signal.signal[signal.SIGUSR1, my_signal_handler];
while[1]:
print["Wait..."]
time.sleep[10]Run the program in the background [with &] and send signals to the
process: $ ./test_signal.py &
[1] 24078
$ Wait...
$ kill -INT 24078
Signal received 2:
$ kill -USR1 24078
Signal received 10:
$ kill -9 24078 REFERENCES & RESOURCES - The Python's mother site @ www.python.org; "The Python Documentation" @ //www.python.org/doc/; "The Python Tutorial" @ //docs.python.org/tutorial/; "The Python Language Reference" @
//docs.python.org/reference/.
- Vernon L. Ceder, "The Quick Python Book", 2nd ed, 2010, Manning [Good starting guide for experience programmers who wish to learning Python].
- Mark Lutz, "Learning Python", 4th ed, 2009; "Programming Python", 4th ed, 2011; "Python Pocket Reference", 4th ed, 2010, O'reilly.
Latest version tested: Python [Ubuntu, Windows, Cygwin, Jupyter Notebook] 3.7.1 and
2.7.14 Last modified: November, 2018
|