TEORIE E TECNICHE DEL RICONOSCIMENTO

Python

Debugging

Debugging

• Uno dei piu’ importanti requisiti per un buon programmatore e’ la capacita’ di identificare gli errori (BUGS) in un programma: DEBUGGING

• Il debugging puo’ essere considerato una forma di– Lavoro investigativo– Esperimento scientifico (si formula un’ipotesi sulla

fonte dell’errore, la si scopre)

IDEE BASE

Two key ideas

1. The scientific method2. Divide and conquer

If you master those, you will find debugging easy, and possibly enjoyable

The scientific method

1. Create a hypothesis2. Design an experiment to test that hypothesis

– Ensure that it yields insight

3. Understand the result of your experiment– If you don’t understand, then possibly suspend your main line of work to

understand that

Tips:• Be systematic

– Never do anything if you don't have a reason– Don’t just flail

• Random guessing is likely to dig you into a deeper hole

• Don’t make assumptions (verify them)

Example experiments

1. An alternate implementation of a function– Run all your test cases afterward

2. A new, simpler test case– Examples: smaller input, or test a function in

isolation– Can help you understand the reason for a failure

Your scientific notebook

Record everything you do• Specific inputs and outputs (both expected and actual)• Specific versions of the program

– If you get stuck, you can return to something that works– You can write multiple implementations of a function

• What you have already tried• What you are in the middle of doing now

– This may look like a stack!• What you are sure of, and why

Your notebook also helps if you need to get help or reproduce your results

Divide and conquer• Where is the defect (or “bug”)?• Your goal is to find the one place that it is• Finding a defect is often harder than fixing it

• Initially, the defect might be anywhere in your program– It is impractical to find it if you have to look everywhere

• Idea: bit by bit reduce the scope of your search• Eventually, the defect is localized to a few lines or one line

– Then you can understand and fix it

• 4 ways to divide and conquer:– In the program code– In test cases– During the program execution– During the development history

Divide and conquer in the program code

• Localize the defect to part of the program– e.g., one function, or one part of a function

• Code that isn’t executed cannot contain the defect

3 approaches:• Test one function at a time• Add assertions or print statements

– The defect is executed before the failing assertion (and maybe after a succeeding assertion)

• Split complex expressions into simpler onesExample: Failure in result = set({graph.neighbors(user)})Change it to nbors = graph.neighbors(user) nbors_set = {nbors} result = set(nbors_set)The error occurs on the “nbors_set = {nbors}" line

Divide and conquer in test cases

• Your program fails when run on some large input– It’s hard to comprehend the error message– The log of print statement output is overwhelming

• Try a smaller input– Choose an input with some but not all

characteristics of the large input– Example: Unicode characters, duplicates, zeroes

in data, …

Divide and conquer in execution timevia print (or “logging”) statements

• A sequence of print statements is a record of the execution of your program

• The print statements let you see and search multiple moments in time

• Print statements are a useful technique, in moderation• Be disciplined

– Too much output is overwhelming rather than informative– Remember the scientific method: have a reason (a

hypothesis to be tested) for each print statement– Don’t only use print statements

Divide and conquerin development history

• The code used to work (for some test case)• The code now fails• The defect is related to some line you changed

• This is useful only if you kept a version of the code that worked (use good names!)

• This is most useful if you have made few changes• Moral: test often!– Fewer lines to compare– You remember what you were thinking/doing recently

Simple Debugging Tools

print– shows what’s happening whether there’s a problem or not– does not stop execution

assert– Raises an exception if some condition is not met– Does nothing if everything works– Example: assert len(rj.edges()) == 16– Use this liberally! Not just for debugging!

raw_input– Stops execution – (Designed to accept user input, but I rarely use it for this.)

TIPI DI ERRORI

Tipi di errori

• Sintattici>>> def faa ()SyntaxError: invalid syntax

• Run time– exceptions

• Semantic– Il programma gira ma non fa quel che deve fare

ERRORI SINTATTICI

• Prodotti da Python quando cerca di tradurre il source code in codice macchina

• Facili da rimediare se si sa dove sono • A volte pero’ gli errori sono del tipo SyntaxError: invalid syntax SyntaxError: invalid token

COME RIMEDIARE AGLI ERRORI SINTATTICI

(i) Make sure you are not using a Python keyword for a variable name. (ii) Check that you have a colon at the end of the header of every compound

statement, including for, while, if, and def statements. (iii) Make sure that any strings in the code have matching quotation marks. (iv) If you have multiline strings with triple quotes (single or double), make sure you

have terminated the string properly. An unterminated string may cause an invalid tokenerrorattheendofyourprogram,oritmaytreatthefollowing part of the program as a string until it comes to the next string. In the second case, it might not produce an error message at all!

(v) An unclosed opening operator – (, {, or [ – makes Python continue with the next line as part of the current statement. Generally, an error occurs almost immediately in the next line.

(vi) Check for the classic = instead of == inside a conditional. (vii) Check the indentation to make sure it lines up the way it is supposed to.Python

can handle space and tabs, but if you mix them it can cause problems. The best way to avoid this problem is to use a text editor that knows about Python and generates consistent indentation.

‘CONTINUO A MODIFICARE IL PROGRAMMA MA NON SUCCEDE NIENTE…’

• A volte il motivo per cui Python vi dice che c’e’ un errore ma voi non lo vedete e’ che Python sta cercando di eseguire una versione del documento diversa da quella che state modificando :– Avete modificato il file ma non lo avete salvato– Avete cambiato il nome del file ma state ancora

caricando il vecchio file– Avete dato al vostro programma il nome di uno dei

moduli di Python – Etc, etc

ERRORI RUNTIME

• Prodotti dall’interprete quando qualcosa va male durante l’esecuzione

ESEMPI DI ERRORI RUNTIME

• Il mio programma non fa nulla– Succede per esempio quando definite classi e

funzioni ma poi non le chiamate• Il mio programma parte ma poi rimane

sospeso (‘hanging’)– Loop infinito– Ricorsione infinita

LOOP INFINITO

x = 1

y = -1

while x > 0 and y < 0 :

x += 1

y -= 1

LOOP INFINITO

x = 1

y = -1

while x > 0 and y < 0 :

x += 1

y -= 1

print "x: ", x

print "y: ", y

print "condition: ", (x > 0 and y < 0)

Una soluzione: usare print

RICORSIONE INFINITA

Maximum recursion depth exceeded error

Soluzioni:- Controllare che ci sia un caso base- Usare print

EXCEPTIONS Traceback (most recent call last): File "nx_error.py", line 41, in <module> print friends_of_friends(rj, myval) File "nx_error.py", line 30, in friends_of_friends f = friends(graph, user) File "nx_error.py", line 25, in friends return set(graph.neighbors(user))# File "/Library/Frameworks/…/graph.py", line 978, in neighbors return list(self.adj[n])TypeError: unhashable type: 'list'

List of all exceptions (errors):http://docs.python.org/2/library/exceptions.html#bltin-exceptionsTwo other resources, with more details about a few of the errors:http://inventwithpython.com/appendixd.htmlhttp://www.cs.arizona.edu/people/mccann/errors-python

Call stack or traceback

First function that was called (<module> means the interpreter)

Second function that was called

Last function that was called (this one suffered an error)

The error message:daunting but useful.You need to understand:• the literal meaning of

the error• the underlying

problems certain errors tend to suggest

ERRORI RUNTIME PIU’ COMUNI• NameError: State usando una variabile non definita nell’ambiente

corrente (per esempio una variabile locale ad un’altra funzione)• TypeError: Ci sono diverse possibilita’

– You are trying to use a value improperly. Example: indexing a string, list, or tuple with something other than an integer.

– There is a mismatch between the items in a format string and the items passed for conversion. This can happen if either the number of items does not match or an invalid conversion is called for.

– You are passing the wrong number of arguments to a function or method. For methods, look at the method definition and check that the first param- eter is self. Then look at the method invocation; make sure you are invoking the method on an object with the right type and providing the other arguments correctly.

• KeyError: You are trying to access an element of a dictionary using a key that the dictionary does not contain.

ERRORI RUNTIME PIU’ COMUNI

• IndexError: The index you are using to access a list, string, or tuple is greater than its length minus one. Immediately before the site of the error, add a print statement to display the value of the index and the length of the array. Is the array the right size? Is the index the right value?

• AttributeError: You are trying to access an attribute or method that does not exist. Check the spelling! You can use dir to list the attributes that do exist. If an AttributeError indicates that an object has NoneType, that means that it is None. One common cause is forgetting to return a value from a function; if you get to the end of a function without hitting a return statement, it returns None. Another common cause is using the result from a list method, like sort, that returns None.

Soluzione

• Usare pdb (vedi sotto)

Ho inserito cosi’ tanti print …

• … che non capisco piu’ cosa succede– Usate solo un numero minimo di print statements,

quelli necessari per risolvere UN problema!

SEMANTIC ERROR

• Il programma gira ma non fa la cosa giusta – per esempio non aggiunge dati ad un dizionario

• Il tipo di errore piu’ difficile da correggere!

COSA FARE• Is there something the program was supposed to do but which

doesn’t seem to be happening? Find the section of the code that performs that function and make sure it is executing when you think it should.

• Is something happening that shouldn’t? Find code in your program that performs that function and see if it is executing when it shouldn’t.

• Is a section of code producing an effect that is not what you expected? Make sure that you understand the code in question, especially if it involves invocations to functions or methods in other Python modules. Read the documentation for the functions you invoke. Try them out by writing simple test cases and checking the results.

ESPRESSIONI COMPLESSE

self.hands[i].addCard(self.hands[self.findNeighbor(i)].popCard())

Meglio:

neighbor = self.findNeighbor(i)

pickedCard = self.hands[neighbor].popCard()

self.hands[i].addCard(pickedCard)

CONTROLLARE IL VALORE DI UNA FUNZIONE

return self.hands[i].removeMatches()

Meglio:

count = self.hands[i].removeMatches() return count

PDB

Pdb: Python debugger

• Un debugger e’ un programma che permette di studiare un problem a run time per esempio– Visualizzando la stack di esecuzione– Facendo stepping del programma

pdb.pm()>>> import pdb>>> def foo ():

return x + 1>>> def faa ():

return foo()>>> faa()Traceback (most recent call last): File "<pyshell#12>", line 1, in <module> faa() File "<pyshell#11>", line 2, in faa return foo() File "<pyshell#4>", line 2, in foo return x + 1NameError: global name 'x' is not defined>>>>>> pdb.pm() <pyshell#4>(2)foo()

pdb.pm()(Pdb) h

Documented commands (type help <topic>):========================================EOF bt cont enable jump pp run unt a c continue exit l q s until alias cl d h list quit step up args clear debug help n r tbreak w b commands disable ignore next restart u whatisbreak condition down j p return unalias where

Miscellaneous help topics:==========================exec pdb

Undocumented commands:======================retval rv

pdb.pm()

(Pdb) w /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/idlelib/run.py(298)runcode()-> exec code in self.locals <pyshell#12>(1)<module>() <pyshell#11>(2)faa() <pyshell#4>(2)foo()

pdb: tracing and stepping

Now add: pdb.set_trace()

Example program:

# epdb1.py -- experiment with the Python debugger, pdba = "aaa"b = "bbb"c = "ccc"final = a + b + cprint final

To get:

# epdb1.py -- experiment with the Python debugger, pdba = "aaa”pdb.set_trace()b = "bbb"c = "ccc"final = a + b + cprint final

pdb: stepping

Now as you execute the program, Python enters the debugger after the first instruction:

(Pdb)

At this point you can execute one instruction at a time using ‘n’

ESEMPIO

FIND_WORDS

>>> def find_words(text, wordlength, result=[]):... for word in text:... if len(word) == wordlength:... result.append(word)... return result>>> find_words(['omg', 'teh', 'lolcat', 'sitted', 'on', 'teh', 'mat'], 3) [1]['omg', 'teh', 'teh', 'mat']>>> find_words(['omg', 'teh', 'lolcat', 'sitted', 'on', 'teh', 'mat'], 2, ['ur']) [2]['ur', 'on']>>> find_words(['omg', 'teh', 'lolcat', 'sitted', 'on', 'teh', 'mat'], 3) [3]['omg', 'teh', 'teh', 'mat', 'omg', 'teh', 'teh', 'mat']

DEBUGGING FIND_WORDS CON PDB

>>> import pdb>>> find_words(['cat'], 3) # [_first-run]['cat']>>> pdb.run("find_words(['dog'], 3)") # [_second-run]> <string>(1)<module>()(Pdb) step--Call--> <stdin>(1)find_words()(Pdb) argstext = ['dog']wordlength = 3result = ['cat']