< BACKMake Note | BookmarkCONTINUE >
156135250194107072078175030179198180025031194137176049106218111004228146003100134192208091

Executable Object Statements and Built-in Functions

Python provides a number of built-in functions supporting callables and executable objects, including the exec statement. These functions let the programmer execute code objects as well as generate them using the compile() built-in function and are listed in Table 14.6.

Table 14.6. Executable Object Statements and Built-in Functions
Built-in Function or Statement Description
callable(obj) determines if obj is callable; returns 1 if so, 0 otherwise
compile(string, file, type) creates a code object from string of type type; file is where the code originates from (usually set to ?)
eval(obj, globals=globals(), locals=locals()) evaluates obj, which is either a expression compiled into a code object or a string expression; global and/or local namespace dictionaries may also be provided, otherwise, the defaults for the current environment will be used
exec obj execute obj, a single Python statement or set of statements, either in code object or string format; obj may also be a file object (opened to a valid Python script)
input(prompt='') equivalent to eval(raw_input(prompt=''))
intern(string) request intern of string

callable()

callable() is a Boolean function which determines if an object type can be invoked via the function operator ( ( ) ). It returns 1 if the object is callable and 0 otherwise. Here are some sample objects and what callable returns for each type:

						
>>> callable(dir)              # built-in function
1
>>> callable(1)                # integer
0
>>> def foo(): pass
…
>>> callable(foo)              # user-defined function
1
>>> callable('bar')            # string
0
>>> class C: pass
…
>>> callable(C)                # class
1

					

compile()

compile() is a function which allows the programmer to generate a code object on the fly, that is, during run-time. These objects can then be executed or evaluated using the exec statement or eval() built-in function. It is important to bring up the point that both exec and eval() can take string representations of Python code to execute. When executing code given as strings, the process of byte-compiling such code must occur every time. The compile() function provides a one-time byte-code compilation of code so that the precompile does not have to take place with each invocation. Naturally, this is an advantage only if the same pieces of code are executed more than once. In these cases, it is definitely better to precompile the code.

All three arguments to compile() are required, with the first being a string representing the Python code to compile. The second string, although required, is usually set to the empty string. This parameter represents the file name (as a string) where this code object is located or can be found. Normal usage is for compile() to generate a code object from a dynamically-generated string of Python code—code which obviously does not read from an existing file.

The last argument is a string indicating the code object type. There are three possible values:

'eval' evaluatable expression [to be used with eval()]
'single' single executable statement [to be used with exec]
'exec' group of executable statements [to be used with exec]

Evaluatable Expression
							
>>> eval_code = compile('100 + 200', '', 'eval')
>>> eval(eval_code)
300

						
Single Executable Statement
							
>>> single_code = compile('print "hello world!"', '', 'single')
>>> single_code
<code object ? at 120998, file "", line 0>
>>> exec single_code
hello world!

						
Group of Executable Statements
							
>>> exec_code = compile("""
…   req = input('Count how many numbers? ')
…   for eachNum in range(req):
…       print eachNum 
…   """, '', 'exec')
>>> exec exec_code
Count how many numbers? 6
0
1
2
3
4
5

						

eval()

eval() evaluates an expression, either as a string representation or a pre-compiled code object created via the compile() built-in. This is the first argument to eval(). The second and third parameters, both optional, represent the objects in the global and local namespaces, respectively. If these arguments are not given, they default to objects returned by globals() and locals(), respectively. Take a look at the following example:

						
>>> eval('932')
932
>>> int('932')
932

					

We see that in this case, both eval() and int() yield the same result: an integer with the value 932. The paths they take are somewhat different, however. eval() takes the string in quotes and evaluates it as a Python expression. int() takes a string representation of an integer and converts it to an integer. It just so happens that the string consists exactly of the string 932, which as an expression yields the value 932, and that 932 is also the integer represented by the string "932." Things are not the same, however, when we use a pure string expression:

						
>>> eval('100 + 200')
300
>>> int('100 + 200')
Traceback (innermost last):
  File "<stdin>", line 1, in ?
ValueError: invalid literal for int(): 100 + 200

					

In this case, eval() takes the string and evaluates "100 + 200" as an expression, which, after performing integer addition, yields the value 300. The call to int() fails because the string argument is not a string representation of an integer—there are invalid literals in the string, namely, the spaces and "+" character.

One simple way to envision how the eval() function works is to imagine that the quotation marks around the expression are invisible and think, "If I were the Python interpreter, how would I view this expression?" In other words, how would the interpreter react if the same expression were entered interactively? The output after pressing the RETURN or ENTER key should be the same as what eval() will yield.

exec

Like eval(), the exec statement also executes either a code object or a string representing Python code. Similarly, precompiling oft-repeated code with compile() helps improve performance by not having to go through the bytecode compilation process for each invocation. The exec statement takes exactly one argument, as indicated here with its general syntax:

						
exec
							obj

					

The executed object (obj) can be either a single statement or a group of statements, and either may be compiled into a code object (with "single" or "exec," respectively) or it can be just the raw string. Below is an example of multiple statements being sent to exec as a single string:

						
>>> exec """
…    x = 0
…    print 'x is currently:', x
…    while x < 5:
…       x = x + 1
…       print 'incrementing x to:', x
…    """
x is currently: 0
incrementing x to: 1
incrementing x to: 2
incrementing x to: 3
incrementing x to: 4
incrementing x to: 5

					

Finally, exec can also accept a valid file object to a (valid) Python file. If we take the code in the multi-line string above and create a file called xcount.py, then we could also execute the same code with the following:

						
>>> f = open('xcount.py')        # open the file
>>> exec f                       # execute the file
x is currently: 0
incrementing x to: 1
incrementing x to: 2
incrementing x to: 3
incrementing x to: 4
incrementing x to: 5
>>> exec f                       # try execution again
>>>                              # oops, it failed… why?

					

Note that once execution has completed, a successive call to exec fails. Well, it really doesn't fail. It just doesn't do anything, which may have caught you by surprise. In reality, exec has read all the data in the file and is sitting at the end-of-file (EOF). When exec is called again with the same file object, there is no more code to execute, so it does not do anything, hence the behavior seen above. How do we know that it is at EOF?

We use the file object's tell() method to tell us where we are in the file and then use os.path.getsize() to tell us how large our xcount.py script was. As you can see, there is an exact match:

						
>>> f.tell()                      # where are we in the file?
116
>>> f.close()                     # close the file
>>> from os.path import getsize
>>> getsize('xcount.py')          # what is the file size?
116

					
Using Python to Generate and Execute Python Code Example

We now present code for the loopmake.py script, which is a simple computer-aided software engineering (CASE) that generates and executes loops on-the-fly. It prompts the user for the various parameters (i.e., loop type (while or for), type of data to iterate over [numbers or sequences]), generates the code string, and executes it.

Example 14.1. Dynamically Generating and Executing Python Code (loopmake.py)
								 <$nopage>
001 1  #!/usr/bin/env python
002 2
003 3  dashes = '\n' + '-' * 50        # dashed line
004 4  exec_dict = {
005 5  
006 6  'f': """                        # for loop
007 7  for %s in %s:
008 8      print %s
009 9  """,
010 10
011 11 's': """                        # sequence while loop
012 12 %s = 0
013 13 %s = %s
014 14 while %s < len(%s):
015 15      print %s[%s]
016 16      %s = %s + 1
017 17 """,
018 18
019 19 'n': """                        # counting while loop
020 20 %s = %d
021 21 while %s < %d:
022 22      print %s
023 23      %s = %s + %d
024 24 """
025 25 }
026 26 
027 27 def main():
028 28 
029 29     ltype = raw_input('Loop type? (For/While) ')
030 30     dtype = raw_input('Data type? (Number/Seq) ')
031 31
032 32     if dtype == 'n':
033 33         start = input('Starting value? ')
034 34         stop = input('Ending value (non-inclusive)? ')
035 35         step = input('Stepping value? ')
036 36         seq = str(range(start, stop, step))
037 37
038 38     else: <$nopage>
039 39         seq = raw_input('Enter sequence: ')
040 40
041 41     var = raw_input('Iterative variable name? ')
042 42
043 43     if ltype == 'f':
044 44         exec_str = exec_dict['f'] % (var, seq, var)
045 45
046 46     elif ltype == 'w':
047 47         if dtype == 's':
048 48             svar = raw_input('Enter sequence name? ')
049 59             exec_str = exec_dict['s'] % \
050 50     (var, svar, seq, var, svar, svar, var, var, var)
051 51
052 52         elif dtype == 'n':
053 53             exec_str = exec_dict['n'] % \
054 54     (var, start, var, stop, var, var, var, step)
055 55
056 56     print dashes
057 57     print 'Your custom-generated code:' + dashes
058 58     print exec_str + dashes
059 59     print 'Test execution of the code:' + dashes
060 60     exec exec_str
061 61     print dashes
062 62
063 63 if __name__ == '__main__':
064 64     main()
065  <$nopage>
							

Here are a few example executions of this script:

								
% loopmake.py
Loop type? (For/While) f
Data type? (Number/Sequence) n
Starting value? 0
Ending value (non-inclusive)? 4


Stepping value? 1
Iterative variable name? counter


--------------------------------------------------
The custom-generated code for you is:
--------------------------------------------------


for counter in [0, 1, 2, 3]:
    print counter


--------------------------------------------------
Test execution of the code:
--------------------------------------------------
0
1
2
3


--------------------------------------------------
% loopmake.py
Loop type? (For/While) w
Data type? (Number/Sequence) n
Starting value? 0
Ending value (non-inclusive)? 4
Stepping value? 1
Iterative variable name? counter


--------------------------------------------------
Your custom-generated code:
--------------------------------------------------


counter = 0
while counter < 4:
    print counter
    counter = counter + 1


--------------------------------------------------
Test execution of the code:
--------------------------------------------------
0
1
2
3


--------------------------------------------------


% loopmake.py
Loop type? (For/While) f
Data type? (Number/Sequence) s
Enter sequence: [932, 'grail', 3.0, 'arrrghhh']
Iterative variable name? eachItem


--------------------------------------------------
Your custom-generated code:
--------------------------------------------------


for eachItem in [932, 'grail', 3.0, 'arrrghhh']:
    print eachItem


--------------------------------------------------
Test execution of the code:
--------------------------------------------------
932
grail
3.0
arrrghhh


--------------------------------------------------
% loopmake.py
Loop type? (For/While) w
Data type? (Number/Sequence) s
Enter sequence: [932, 'grail', 3.0, 'arrrghhh']
Iterative variable name? eachIndex
Enter sequence name? myList


--------------------------------------------------
Your custom-generated code:
--------------------------------------------------


eachIndex = 0
myList = [932, 'grail', 3.0, 'arrrghhh']
while eachIndex < len(myList):
    print myList[eachIndex]
    eachIndex = eachIndex + 1


--------------------------------------------------
Test execution of the code:
--------------------------------------------------
932
grail
3.0
arrrghhh


--------------------------------------------------

							
Line-by-line Explanation
Lines 1 – 25

In this first part of the script, we are setting up two global variables. The first is a static string consisting of a line of dashes (hence the name) and the second is a dictionary of the skeleton code we will need to use for the loops we are going to generate. The keys are "f" for a for loop, "s" for a while loop iterating through a sequence, and "n" for a counting while loop.

Lines 27 – 30

Here we prompt the user for the type of loop he or she wants and what data types to use.

Lines 32 – 36

Numbers have been chosen; they provide the starting, stopping, and incremental values. In this section of code, we are introduced to the input() built-in function for the first time. As we shall see in Section 14.3.5, input() is similar to raw_input() in that it prompts the user for string input, but unlike raw_input(), input() also evaluates the input as a Python expression, rendering a Python object even if the user typed it in as a string.

Lines 38 – 39

A sequence was chosen; enter the sequence here as a string.

Line 41

Get the name of the iterative loop variable that the user wants to use.

Lines 43 – 44

Generate the for loop, filling in all the customized details.

Lines 46 – 50

Generate a while loop which iterates through a sequence.

Lines 52– 54

Generate a counting while loop.

Lines 56 – 61

Output the generated source code as well as the resulting output from execution of the aforementioned generated code.

Lines 63 – 64

Execute main() only if this module was invoked directly.

To keep the size of this script to a manageable size, we had to trim all the comments and error checking from the original script. You can find both the original as well as an alternate version of this script on the CD-ROM in the back of the text.

The extended version includes extra features such as not requiring enclosing quotation marks for string input, default values for input data, and detection of invalid ranges and identifiers; it also does not permit built-in names or keywords as variable names.

input()

The input() built-in function is the same as the composite of eval() and raw_input(), equivalent to eval(raw_input()). Like raw_input(), input() has an optional parameter which represents a string prompt to display to the user. If not provided, the string has a default value of the empty string.

Functionally, input() differs from raw_input() because raw_input() always returns a string containing the user's input, verbatim. input() performs the same task of obtaining user input; however it takes things one step further by evaluating the input as a Python expression. This means that the data returned by input() is a Python object, the result of performing the evaluation of the input expression.

One clear example is when the user inputs a list. raw_input() returns the string representation of a list, while input() returns the actual list:

						
>>> aString = raw_input('Enter a list: ')
Enter a list: [ 123, 'xyz', 45.67 ]
>>> aString
"[ 123, 'xyz', 45.67 ]"
>>> type(aString)
<type 'string'>

					

The above was performed with raw_input(). As you can see, everything is a string. Now let us see what happens when we use input() instead:

						
>>> aList = input('Enter a list: ')
Enter a list: [ 123, 'xyz', 45.67 ]
>>> aList
[123, 'xyz', 45.67]
>>> type(aList)
<type 'list'>

					

Although the user input a string, input() evaluates that input as a Python object and returns the result of that expression.

Interned Strings and intern()

For performance reasons, Python keeps an internal string table consisting of static string literals and identifier strings whose purpose is to speed up dictionary lookups. By keeping these strings around, any time a reference is made to either the same string literal, or to an identifier which bears a name that is in the table, no new space needs to be allocated for that string, hence saving the time it takes for memory allocation. This "interned" set of strings is not deallocated or garbage-collected until the interpreter exits.

Here is an example of an interned string:

						
>>> id('hello world')
1040072
>>> foo = 'hello world'
>>> id(foo)
1040072
>>> del foo
>>> id('hello world')
1040072

					

The string "hello world" is created in the first statement. The call to id() to reveal its identity was not necessary since the string was created regardless of whether or not the call was made. We did so in the example to immediately display its ID once it was created. Upon assigning this string to an identifier, we observe with another call to id() that, indeed, foo is referencing the same string object, which has been interned. If we remove the object, thereby decrementing the reference count, we see that this has no effect on the string which has been interned.

One surprising aspect may be that the string "foo" itself was interned (as the string name of an identifier). If we create two other strings, we can see that the "foo" string has an earlier ID than the newer strings, indicating that it was created first.

						
>>> id('bar')
1053088
>>> id('foo')
1052968
>>> id('goo')
1053728

					

Python 1.5 saw the debut of the intern() built-in function, which lets the programmer explicitly request that a string be interned. The syntax of intern(), as you may suspect, is:

						
intern(string)
					

The given string argument is the string to intern. intern() enters the string in the (global) table of interned strings. If the string is not already in the interned string table, it is interned and the string is returned. Otherwise, if the string is already there, it is simply returned.


Last updated on 9/14/2001
Core Python Programming, © 2002 Prentice Hall PTR

< BACKMake Note | BookmarkCONTINUE >

© 2002, O'Reilly & Associates, Inc.