Home

1.x PDF - Read the Docs

1. T U NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE 9 3 2 About the IPython Development Team Fernando Perez began IPython in 2001 based on code from Janko Hauser lt jhauser AT zscout de gt and Nathaniel Gray lt n8gray AT caltech edu gt Fernando is still the project lead The Python Development Team is the set of all contributors to the Python project This includes all of the Python subprojects Here is a list of the currently active contributors e Matthieu Brucher 424 Chapter 9 About IPython IPython Documentation Release 1 2 1 An Afternoon Hack Ondrej Certik Laurent Dufrechou Robert Kern Thomas Kluyver Brian E Granger Paul Ivanov Evan Patterson Fernando Perez project leader Benjamin Ragan Kelley Ville M Vainio Gael Varoququx Stefan van der Walt Barry Wark If your name is missing please add it 9 3 3 Our Copyright Policy Python uses a shared copyright model Each contributor maintains copyright over their contributions to IPython But it is important to note that these contributions are typically only changes diffs commits to the repositories Thus the Python source code in its entirety is not the copyright of any
2. 20000 10000 0 10000 20000 30000 5000 10000 15000 20000 25000 30000 In 35000 Spectrogram 6000 8000 10000 12000 14000 16000 The notebook handles very long outputs much better than before this was a serious usability issue when running processes that generated massive amounts of output Now in the presence of outputs longer than 100 lines the notebook will automatically collapse to a scrollable area and the entire left part of this area controls the display one click in this area will expand the output region completely and a double click will hide it completely This figure shows both the scrolled and hidden modes 72 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack In 3 for i in range 2e print i b click to unscroll output double click to hide 11 12 13 14 15 16 17 18 19 a In 3 for i in range 2900 print i Note The auto folding of long outputs is disabled in Firefox due to bugs in its scrolling behavior See PR 2047 for details Uploading notebooks to the dashboard is now easier in addition to drag and drop which can be finicky sometimes you can now click on the upload text and use a regular file di alog box to select notebooks to upload Furthermore the notebook dashboard now auto refreshes its contents and offers buttons to shut down any running kernels PR 1739 Note
3. c NotebookApp notebook_manager_class TPython html services notebooks azurenbmanager AzureNotebookManager c AzureNotebookManager account_name u paste_your_account_name_here c AzureNotebookManager account_key u paste_your_account_key_here c AzureNotebookManager container u notebooks In addition to providing your Azure Blob Storage account name and key you will have to provide a container name you can use multiple containers to organize your notebooks 4 8 5 Known issues When behind a proxy especially if your system or browser is set to autodetect the proxy the notebook web application might fail to connect to the server s websockets and present you with a warning at startup In this case you need to configure your system not to use the proxy for the server s address For example in Firefox go to the Preferences panel Advanced section Network tab click Settings and add the address of the notebook server to the No proxy for field 4 8 Running a notebook server 257 IPython Documentation Release 1 2 1 An Afternoon Hack 258 Chapter 4 Using IPython for interactive work CHAPTER 5 Using IPython for parallel computing 5 1 Overview and getting started 5 1 1 Examples We have various example scripts and notebooks for using Python parallel in our examples parallel directory or they can be found on GitHub Some of these are covered in more detail in the examples sec
4. 2 execute ZeroDivisionError Traceback most recent call last gt 1 1 0 ZeroDivisionError integer division or modulo by zero 3 execute ZeroDivisionError Traceback most recent call last gt 1 1 0 ZeroDivisionError integer division or modulo by zero Notice how the error message printed when CompositeError is raised has information about the indi vidual exceptions that were raised on each engine If you want you can even raise one of these original exceptions In 80 try aans dview execute 1 0 block True 1 except parallel error CompositeError e R es e raise_exception ZeroDivisionError Traceback most recent call last gt 1 1 0 ZeroDivisionError integer division or modulo by zero If you are working in Python you can simple type debug after one of these CompositeError excep tions is raised and inspect the exception instance In 81 dview execute 1 0 O execute ZeroDivisionError Traceback most recent call last gt 1 1 0 ZeroDivisionError integer division or modulo by zero l execute ZeroDivisionError Traceback most recent call last gt 1 1 0 ZeroDivisionError integer division or modulo by zero 2 execute 5 3 IPython s Direct interface 287 IPython Documentation Release 1 2 1 An Afternoon Hack ZeroDivisionError Traceback most recent
5. 5 7 2 Starting the engines with MPI enabled To use code that calls MPI there are typically two things that MPI requires 1 The process that wants to call MPI must be started using mpiexec or a batch system like PBS that has MPI support 2 Once the process starts it must call MPI_Init There are a couple of ways that you can start the Python engines and get these things to happen Automatic starting using mpiexec and ipcluster The easiest approach is to use the MPI Launchers in ipcluster which will first start a controller and then a set of engines using mpiexec ipcluster start n 4 ngines MPIEngineSetLauncher This approach is best as interrupting ipcluster will automatically stop and clean up the controller and engines 306 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack Manual starting using mpiexec If you want to start the Python engines using the mpiexec just do mpiexec n 4 ipengin mpi mpi4py This requires that you already have a controller running and that the FURL files for the engines are in place We also have built in support for PyTrilinos Py7rilinos which can be used assuming is installed by starting the engines with mpiexec n 4 ipengin mpi pytrilinos Automatic starting using PBS and ipcluster The ipcluster command also has built in integration with PBS For more informatio
6. PR 1828 PR 1800 PR 1830 PR 1773 PR 1761 PR 1822 PR 1821 PR 1820 PR 1816 PR 1813 PR 1711 PR 1798 PR 1812 PR 1811 PR 1810 PR 1808 PR 1742 PR 1807 PR 1805 PR 1739 PR 1770 PR 1707 PR 1797 PR 1795 PR 1793 PR 1794 PR 1792 PR 1791 PR 1787 PR 1758 PR 1748 PR 1755 change default extension to ipy for save r Reintroduce recall Ismagic lists magics in alphabetical order Update SymPy profile SymPy s latex can now print set and frozenset Edited documentation to use IPYTHONDIR in place of ipython aesthetics pass on AsyncResult display_outputs ENTER submits the rename notebook dialog NotebookApp Make the number of ports to retry user configurable Always use filename as the notebook name Add assert_in method to nose for Python 2 6 New Tooltip New Completer and JS Refactor a few simple fixes for docs parallel Ensure AsyncResult display_outputs doesn t display empty streams warn on nonexistent exclusions in iptest fix for 1809 failing tests in Python zmq Reposition alternate upload for firefox need cross browser OS language test Check for custom_exceptions only once add missing cython exclusion in iptest Fixed a vevarsall bat error on win32 Py2 7 when trying to compile with m Dashboard improvement necessary merge of 1658 and 1676 fix 1492 Cython related magic functions Accept gui lt gt switch in Python qtconsole
7. The parallel tools now default to using NoDB as the storage backend for intermediate results This means that the default usage case will have a significantly reduced memory footprint though certain advanced features are not available with this backend For more details see Python s Task Database The parallel magics now display all output so you can do parallel plotting or other actions with complex display The px magic has now both line and cell modes and in cell mode finer control has been added about how to collate output from multiple engines PR 1768 There have also been incremental improvements to the SSH launchers e add to_send fetch steps for moving connection files around 84 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e add SSHProxyEngineSetLauncher for invoking to ipcluster engines on a remote host This can be used to start a set of engines via PBS SGE MPI remotely This makes the SSHLauncher usable on machines without shared filesystems A number of sugar methods properties were added to AsyncResult that are quite useful PR 1548 for everday work e ar wall_time received submitted e ar serial_time sum of serial computation time e ar elapsed time since submission wall_time if done e ar progress int number of sub tasks that have completed e len ar of tasks e ar wait_interactive prints progress Added Client spin_thread
8. e Ctrl m This is the prefix for all other shortcuts which consist of Ct r1 m followed by a single letter or character For example if you type Ctrl m h that is the sole letter h after Ctr1 m Python will show you all the available keyboard shortcuts Here is the complete set of keyboard shortcuts available 4 6 The IPython Notebook 249 IPython Documentation Release 1 2 1 An Afternoon Hack Shortcut Action Shift Enter run cell Ctrl Enter run cell in place Alt Enter run cell insert below Ctrl m x cut cell Ctrl m c copy cell Ctrl m v paste cell Ctrl m d delete cell Ctrl m z undo last cell deletion Ctrl m split cell Ctrl m a insert cell above Ctrl m b insert cell below Ctrl m o toggle output Ctrl m O toggle output scroll Ctrl m toggle line numbers Ctrl m s save notebook Ctrl m j move cell down Ctrl m k move cell up Ctrl m y code cell Ctrl m m markdown cell Ctrl m t raw cell Ctrl m 1 6 heading 1 6 cell Ctrl m p select previous Ctrl m n select next Ctrl m i interrupt kernel Ctrl m restart kernel Ctrl m h show keyboard shortcuts 4 6 6 Plotting One major feature of the notebook is the ability to display plots that are the output of running code cells Python is designed to work seamlessly with the matplotlib plotting library to provide this functionality To set this up before any
9. gt 1 a b NameError name b is not defined 3 more exceptions 288 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack All of this same error handling magic even works in non blocking mode In 83 dview block False In 84 ar dview execute 1 0 In 85 ar get O execute ZeroDivisionError Traceback most recent call last gt 1 1 0 ZeroDivisionError integer division or modulo by zero 3 more exceptions 5 4 Parallel Magic Commands We provide a few Python magic commands that make it a bit more pleasant to execute Python com mands on the engines interactively These are mainly shortcuts to DirectView execute and AsyncResult display_outputs methods repsectively These magics will automatically become available when you create a Client In 2 re parallel Client The initially active View will have attributes targets all block True which is a blocking view of all engines evaluated at request time adding removing engines will change where this view s tasks will run 5 4 1 The Magics PX The px magic executes a single Python command on the engines specified by the targets attribute of the Direct View instance import numpy here and everywher In 25 with re sync_imports wb eet import numpy importing numpy on engine s In 27
10. jobs node randomwait 5 12 DAG Dependencies 329 des and edg IPython Documentation Release 1 2 1 An Afternoon Hack Once we have a dict of jobs matching the nodes on the graph we can start submitting jobs and linking up the dependencies Since we don t know a job s msg_id until it is submitted which is necessary for building dependencies it is critical that we don t submit any jobs before other jobs it may depend on Fortunately NetworkX provides a topological_sort method which ensures exactly this It presents an iterable that guarantees that when you arrive at a node you have already visited all the nodes it on which it depends In 5 re Client In 5 view rc load_balanced_view In 6 results In 7 for node in G topological_sort get list of AsyncResult objects from nodes leading into this one as dependencies deps results n for n in G predecessors node submit and store AsyncResult object with view temp_flags after deps block False results node view apply_with_flags jobs node Now that we have submitted all the jobs we can wait for the results In 8 view wait results values Now at least we know that all the jobs ran and did not fail r get would have raised an error if a task failed But we don t know that the ordering was properly respected For this we can use the metadata attribute of eac
11. Out 6 43 The module was reloaded without reloading it explicitly and the object imported with from foo import was also updated Usage The following magic commands are provided 356 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack sautoreload Reload all modules except those excluded by Saimport automatically now sautoreload 0 Disable automatic reloading sautoreload 1 Reload all modules imported with saimport every time before executing the Python code typed Sautoreload 2 Reload all modules except those excluded by Saimport every time before executing the Python code typed Saimport List modules which are to be automatically imported or not to be imported Saimport foo Import module foo and mark it to be autoreloaded for Sautoreload 1 Saimport foo Mark module foo to not be autoreloaded Caveats Reloading Python modules in a reliable way is in general difficult and unexpected things may occur autoreloadtries to work around common pitfalls by replacing function code objects and parts of classes previously in the module with new versions This makes the following things to work e Functions and classes imported via from xxx import foo are upgraded to new versions when xxx is reloaded e Methods and properties of classes are upgraded on reload so that calling c fooQ on an object c created be
12. c MPI use mpidpy Unfortunately even this won t work for some MPI implementations If you are having problems with this you will likely have to use a custom Python executable that itself calls MPI_Init at the appropriate time Fortunately mpi4py comes with such a custom Python executable that is easy to install and use However this custom Python executable approach will not work with ipcluster currently More details on using MPI with Python can be found here Using ipcluster in PBS mode The PBS mode uses the Portable Batch System PBS to start the engines As usual we will start by creating a fresh profile ipython profile create parallel profile pbs And in ipcluster_config py we will select the PBS launchers for the controller and engines c IPClusterStart controller_launcher_class PBSControllerLauncher c IPClusterEngines engine_launcher_class PBSEngineSetLauncher 268 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack Note Note that the configurable is PClusterEngines for the engine launcher and PClusterStart for the controller launcher This is because the start command is a subclass of the engine command adding a controller launcher Since it is a subclass any configuration made in I PClusterEngines is inherited by IPClusterStart unless it is overridden Python does pro
13. id np str Datatype not understood Autocompletion of object attributes in arrays Reset locale setting in qtconsole python3 3 Entry Point not found UnicodeDecodeError when using Image data some jpeg url regexp only finds one link qtconsole breaks doctest testmod in Python 3 3 SIGUSRI not available on Windows registration purging stalled registration high occurrence in small clusters diff ability of notebooks Crash with pygit2 Bug handling Ellipsis NB css inconsistent behavior between ff and webkit unicode errors when opening a new notebook Installning ipython qtConsole should be easyer att Windows Profile creation fails on 0 13 2 branch docstring typo inconsistency mention of an xml notebook format HDFS5 library segfault possibly due to mismatching headers In notebook importing sympy closes ipython kernel f __globals__ causes an error in Python 3 3 Failing test test_interactiveshell TestAstTransform on Windows alt text for click to expand output has typo in alt text history to print all input history of a previous session when line range is omitted Python installed within virtualenv WARNING Please install Python inside the virtualtenv Completion in Emacs Python buffer causes prompt to be increased Ctrl C finishes notebook immediately cython_pyximport reload broken in python3 Incompatible Qt imports when running inprocess_qtconsole IPython 0 13 1 The check of PyQt version is w
14. 1 Use ipcluster to start 15 engines We used 16 cores of an SGE linux cluster 1 controller 15 engines 2 With the file parallelpi py in your current working directory open up IPython enable mat plotlib and type run parallelpi py This will download the pi files via ftp the first time you run it if they are not present in the Engines working directory When run on our 16 cores we observe a speedup of 14 2x This is slightly less than linear scaling 16x because the controller is also running on one of the cores To emphasize the interactive nature of Python we now show how the calculation can also be run by simply typing the commands from parallelpi py interactively into Python In 1 from IPython parallel import Client The Client allows us to use th ngines interactively We simply pass Client the name of the cluster profile we are using In 2 Client profile mycluster Tn 3 v ols In 4 run pidigits py In 5 filestring pi200m ascii i 02do0f20 Create the list of files to process o In 6 files filestring i i for i in range 1 16 In 7 files Out 7 pi200m ascii Olof20 pi200m ascii 020f20 pi200m ascii O030f20 324 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack pi200m ascii 040f20 pi200m ascii 050f20 pi200m ascii 060f20 pi200m ascii O
15. 314 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack 5 10 Getting started with Windows HPC Server 2008 5 10 1 Introduction The Python programming language is an increasingly popular language for numerical computing This is due to a unique combination of factors First Python is a high level and interactive language that is well matched to interactive numerical work Second it is easy often times trivial to integrate legacy C C Fortran code into Python Third a large number of high quality open source projects provide all the needed building blocks for numerical computing numerical arrays NumPy algorithms SciPy 2D 3D Visualization matplotlib Mayavi Chaco Symbolic Mathematics Sage Sympy and others The Python project is a core part of this open source toolchain and is focused on creating a comprehensive environment for interactive and exploratory computing in the Python programming language It enables all of the above tools to be used interactively and consists of two main components e An enhanced interactive Python shell with support for interactive plotting and visualization e An architecture for interactive parallel computing With these components it is possible to perform all aspects of a parallel computation interactively This type of workflow is particularly relevant in scientific and numerical computing where algorithms code and data are continuall
16. 5 14 2 Creating a Client Creating a client with default settings has not changed much though the extended options have One sig nificant change is that there are no longer multiple Client classes to represent the various execution models There is just one low level Client object for connecting to the cluster and View objects are created from that Client that provide the different interfaces for execution To create a new client and set up the default direct and load balanced objects old In 1 from IPython kernel import client as kclient In 2 mec kclient MultiEngineClient In 3 te kclient TaskClient HE B oO z In 1 from IPython parallel import Client 342 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack In 2 re Client In 3 dview rc In 4 lbview rc load_balanced_view 5 14 3 Apply The main change to the API is the addition of the apply to the View objects This is a method that takes view apply f args kwargs and calls f args kwargs remotely on one or more engines returning the result This means that the natural unit of remote execution is no longer a string of Python code but rather a Python function e non copying sends track e remote References The flags for execution have also changed Previously there was only block denoting whether to wait for resul
17. In 1 Smatplotlib 5 11 1 150 million digits of pi In this example we would like to study the distribution of digits in the number pi in base 10 While it is not known if pi is a normal number a number is normal in base 10 if 0 9 occur with equal likelihood numerical investigations suggest that it is We will begin with a serial calculation on 10 000 digits of pi and then perform a parallel calculation involving 150 million digits In both the serial and parallel calculation we will be using functions defined in the pidigits py file which is available in the examples parallel directory of the Python source distribution These func tions provide basic facilities for working with the digits of pi and can be loaded into Python by putting pidigits py in your current working directory and then doing In 1 run pidigits py 5 11 Parallel examples 321 IPython Documentation Release 1 2 1 An Afternoon Hack Serial calculation For the serial calculation we will use SymPy to calculate 10 000 digits of pi and then look at the frequencies of the digits 0 9 Out of 10 000 digits we expect each digit to occur 1 000 times While SymPy is capable of calculating many more digits of pi our purpose here is to set the stage for the much larger parallel calculation In this example we use two functions from pidigits py one_digit_fregqs which calculates how many times each digit occurs and plot_one_digit_fr
18. Some parallel workloads can be described as a Directed Acyclic Graph or DAG See DAG Dependencies for an example demonstrating how to use map a NetworkX DAG onto task dependencies Impossible Dependencies The schedulers do perform some analysis on graph dependencies to determine whether they are not possible to be met If the scheduler does discover that a dependency cannot be met then the task will fail with an ImpossibleDependency error This way if the scheduler realized that a task can never be run it won t sit indefinitely in the scheduler clogging the pipeline The basic cases that are checked 5 5 The IPython task interface 299 IPython Documentation Release 1 2 1 An Afternoon Hack e depending on nonexistent messages e follow dependencies were run on more than one machine and all True e any dependencies failed and all True success True failures False e all dependencies failed and all False success True failure False Warning This analysis has not been proven to be rigorous so it is likely possible for tasks to become impossible to run in obscure situations so a timeout may be a good choice 5 5 5 Retries and Resubmit Retries Another flag for tasks is retries This is an integer specifying how many times a task should be resubmitted after failure This is useful for tasks that should still run if their engine was shutdown or may have some statistical chance of failing The default is to not retry t
19. To see other command line options do ipcluster h 5 2 3 Configuring an IPython cluster Cluster configurations are stored as profiles You can create a new profile with ipython profile create parallel profile myprofile This will create the directory IPYTHONDIR profile_myprofile and populate it with the default configuration files for the three IPython cluster commands Once you edit those files you can continue to call 266 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack ipcluster ipcontroller ipengine with no arguments beyond profile myprofile and any configuration will be maintained There is no limit to the number of profiles you can have so you can maintain a profile for each of your common use cases The default profile will be used whenever the profile argument is not specified so edit IPYTHONDIR profile_default x_config py to represent your most common use case The configuration files are loaded with commented out settings and explanations which should cover most of the available possibilities Using various batch systems with ipcluster ipcluster has a notion of Launchers that can start controllers and engines with various remote execu tion schemes Currently supported models include ssh mpiexec PBS style Torque SGE LSF and Windows HPC Server In general these are configured by the IPClusterEngines en
20. or as the first character of a line For example 4 3 IPython reference 219 IPython Documentation Release 1 2 1 An Afternoon Hack In 1 my_function home m becomes my_function home me If you use the whole argument is quoted as a single string while splits on whitespace In 2 my_function a b c becomes my_function a b c In 3 my_function a b c becomes my_function a b c Note that the or MUST be the first character on the line This won t work In 4 x my_function home me syntax error 4 3 3 IPython as your default Python environment Python honors the environment variable PYTHONSTARTUP and will execute at startup the file referenced by this variable If you put the following code at the end of that file then Python will be your working environment anytime you start Python from IPython frontend terminal ipapp import launch_new_instance launch_new_instance raise SystemExit The raise SystemExit is needed to exit Python when it finishes otherwise you ll be back at the normal Python gt gt gt prompt This is probably useful to developers who manage multiple Python versions and don t want to have corre spondingly multiple Python versions Note that in this mode there is no way to pass Python any command line options as those are trapped first by Python itself 4 3 4 E
21. 3155 3045 3146 3145 2385 3131 3139 3127 3129 3123 3121 3087 3084 3113 3110 Request Enable cell magics inside of ipy scripts Multiline string does not work includes with doctest type input in IPython notebook Request Line breaks in line magic command poor parallel performance with many jobs print inside multiprocessing crashes Ipython kernel Bad md5 hash for package 0 13 2 Notebook Ipython Kernel does not start if disconnected from internet network Using celery in python 3 3 The notebook viewer is down grep color not working well with notebook Quickly install from source in a clean virtualenv Rolling log for ipython notebook with pylab inline appears to call figure draw twice Walking up and down the call stack Notebook crashed if unplugged ethernet cable NB should use normalize css was 3049 Disable spellchecking in notebook ipython pyqt 4 10 incompatibilty QTextBlockUserData Fails to install under Jython 2 7 beta Render of h4 headers is not correct in notebook error in renderedhtml css 60 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 3109 3107 3103 3090 3081 2834 3091 3074 2996 3065 3067 3061 3049 3039 3048 3042 3035 3031 2991 3027 3020 3023 2963 3018 2484 3014 3007 2955 3006 3005 2940 3000 BUG read_csv dtype
22. 5 12 DAG Dependencies 327 IPython Documentation Release 1 2 1 An Afternoon Hack 328 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack import networkx as nx G nx DiGraph add 5 nodes labeled 0 4 map G add_node range 5 1 2 depend on 0 add_edge 0 1 add_edge 0 2 3 depends on 1 2 add_edge 1 3 add_edge 2 3 4 depends on 1 add_edge 1 4 Qe OQO MM now draw the graph pos 0 0 0 1 2 G 4 2 1 0 33 0 2 4 2 2 3 nx draw G pos edge_color r For demonstration purposes we have a function that generates a random DAG with a given number of nodes and edges def random_dag nodes edges Generate a random Directed Acyclic Graph DAG with a given number of nd G nx DiGraph for i in range nodes G add_node i while edges gt 0 a randint 0 nodes 1 b a while b a b randint 0 nodes 1 G add_edge a b if nx is_directed_acyclic_graph G edges 1 else we closed a loop G remove_edge a b return G So first we start with a graph of 32 nodes with 128 edges In 2 G random_dag 32 128 Now we need to build our dict of jobs corresponding to the nodes on the graph In 3 jobs in reality each job would presumably be different randomwait is just a function that sleeps for a random interval In 4 for node in G
23. 788 Progress indicator in the notebook and perhaps the Qt console 1034 Single process Qt console 1557 magic function conflict while using pylab 1476 Pylab figure objects not properly updating 1704 ensure all needed qt parts can be imported before settling for one 1708 test failure in arg_split on windows 1706 Mark test_push_numpy_nocopy as a known failure for Python 3 1696 notebook tooltip fail on function with number 1698 fix tooltip on token with number 1226 Windows GUI only pythonw bug for Python on Python 3 x 1245 pythonw py3k fixes for issue 1226 1417 Notebook Completer Class 1690 Bogus Deliberately make a test fail 1685 Add script to test pull request 1167 Settle on a choice for IPYTHONDIR 1693 deprecate IPYTHON_DIR in favor of IPYTHONDIR 1672 ipython qtconsole desktop is using a deprecated format 1695 Avoid deprecated warnings from ipython qtconsole desktop 1694 Add quote to notebook to allow it to load 1240 sys path missing as first entry when kernel launched without interface 1689 Fix sys path missing as first entry in ipython kernel 1683 Parallel controller failing with Pymongo 2 2 1687 import Binary from bson instead of pymongo 1614 Display Image in Qtconsole 1616 Make IPython core display Image less notebook centric 1684 CLN Remove redundant function definition 1655 Add open magic command to open editor in non blocking manner 1677
24. IPython Documentation Release 1 2 1 An Afternoon Hack In 6 dview apply lambda x atb x 27 Out 6 42 42 42 42 You can also select blocking execution on a call by call basis with the appl y_sync method In 7 dview block False In 8 dview apply_sync lambda x atbt x 27 Out 8 42 42 42 42 Python commands can be executed as strings on specific engines by using a View s execute method In 6 re 2 execute c atb In 7 re 1 2 execute c a b In 8 dview c shorthand for dview pull c block True Out 8 15 5 15 5 Non blocking execution In non blocking mode apply submits the command to be executed and then returns a AsyncResult object immediately The AsyncResult object gives you a way of getting a result at a later time through its get method See also Docs on the AsyncResult object This allows you to quickly submit long running commands without blocking your local Python IPython session define our function In 6 def wait t ee aie import time PRESS tic time time PE time sleep t SEE return time time tic In non blocking mode In 7 ar dview apply_async wait 2 Now block for the result In 8 ar get Out 8 2 0006198883056641 1 9997570514678955 1 9996809959411621 2 0003249445233154 Again in non blocking mode In 9 ar dview apply_async wait 10 Poll to see
25. git remote git status 7 2 Messaging in IPython 7 2 1 Introduction This document explains the basic communications design and messaging specification for how the various Python objects interact over a network transport The current implementation uses the ZeroMQ library for messaging within and between hosts Note This document should be considered the authoritative description of the Python messaging protocol and all developers are strongly encouraged to keep it updated as the implementation evolves so that we have a single common reference for all protocol details 7 2 Messaging in IPython 383 IPython Documentation Release 1 2 1 An Afternoon Hack The basic design is explained in the following diagram 1 2 3 4 5 6 7 8 9 0 lQ WEIRIT Y U T o P A S D F G H J K L ZIX CVB N MENTER l DEAL SUB A 1123145161781910 QWIEIRIITIYIUI TOP A S D FG H JKC Z X C V B N MJENTER DEAL SUB DEA fs ROUTER ROUTER Cuil Kernel raw_inpu Requests to ker Kernel output bi Request Reply c A single kernel can be simultaneously connected to one or more frontends The kernel has three sockets that serve the following functions 1 stdin this ROUTER socket is connected to all frontends and it allows the kernel to request input from the active frontend when raw_input is called The frontend that executed the code
26. 1950 1948 1941 1942 1833 1940 1938 1924 1925 1461 1936 1542 1937 1572 1935 1849 qtconsole take prefix into account for completion fix another FreeBSD HOME symlink issue Fix completion of tim in the Qt console Make it easy to resize jpeg png images back to original size Fix psums example description in docs ctrl z clears cell output in notebook when pressed enough times fix for 1678 undo no longer clears cells avoid duplicate Websockets closed dialog on ws close UnicodeDecodeError on directory with unicode chars in prompt styling prompt color Normal excepts Support unicode prompts Yopage not working on qtconsole for Windows XP 32 bit update to latest version of vim ipython Add proc option to script fix indentation in kernel js move import RemoteError after get_exc_info Fix for copy action Ctrl C when there is no pager defined in qtconsole Fix help string for InteractiveShell ast_node_interactivity script magics cause terminal spam swallow stderr of which in utils process find_cmd completer draws slightly too small on Chrome fix completer css on some Chrome versions remove remaining references to deprecated XREP XREQ names HTML superscripts not shown raised in the notebook Fix styling of superscripts and subscripts Closes 1924 User notification if notebook saving fails increase duration of save messages save magic fails in clients without stdin
27. 3397 _trepr_html_ error Space in long Paragraph Markdown cell with Chinese or Japanese Run Cell Does not Work Last lines in long cells are hidden Revisit serialization in Python parallel Can t connect to 2 HTTPS notebook servers on the same host 678 Input prompt stripping broken with multiline data structures IPython notebook dirty flag is not set when a cell has unsaved changes Multiprocessing semantics in parallel view map links across notebooks Tornado 3 0 update pretty to use Python 3 style for sets Can t escape multiple dollar signs in a markdown cell Image signature doc improvements Bug in Python external path __init__ py Feature suggestion Download matplotlib figure to client browser autoexported notebooks only export explicitly marked cells Notebook Summary table extracted from markdown headers Zooming notebook in chrome is broken in master Implement autosave in notebook Highlighting matching parentheses module search segfault ipcluster version 0 13 2 Ipython genutils py doesn t exist Feature request Save by cell and not by line Python save magic Non Responsive Kernel Running a Django development server from an Python Notebook Save cell toolbar and slide type metadata in notebooks Yopaste regression with blank lines Weird error with variable and grep in command line magic command Key auto completion in dictionaries Codemirror linenumber css broken Vertical text misalignment
28. 439 291 447 444 445 428 434 441 432 409 402 Backgrounded Tasks not Allowed but easy to slip by Shouldn t use pexpect for subprocesses in in process terminal frontend magic to return exit status OSX readline detection fails in the debugger tests fail without unicode filename support iptest script has 0 exit code even when tests fail client db_query behaves different with SQLite and MongoDB magic_install_default_config test fails when there is no ipython directory IPYTHON_DIR and IPYTHONDIR don t expand tilde to directory Test machinery is imported at normal runtime Users should be able to use Up Down for cell navigation Users should be able to copy notebook input and output Rename special display methods and put them lower in priority than display functions Instantiating classes without __init__ function causes kernel to crash Ctrl t in WxIPython Causes Unexpected Behavior qt and console Based Startup Errors ipcluster doesn t handle stale pid info well 10 0 2 seg fault with rpy2 Allow running a block of code in a file Silent request fails Test failure in Python lib History section of messaging spec is incorrect 88 Error when inputting UTF8 CJK characters 366 425 304 216 412 411 417 Ctrl K should kill line and store it so that Ctrl y can yank it back typo in gui magic help Persistent warnings if old configuration files exist crash of ipython when alias is
29. 993 raw_input redirection to foreign kernels is extremely brittle 1100 ipython3 traceback unicode issue from extensions 1071 Large html notebooks hang on load on a slow machine 2 7 Issues closed in the 0 12 development cycle 145 IPython Documentation Release 1 2 1 An Afternoon Hack 89 pdoc np ma compress shows docstring twice 22 Include improvements from anythingipython el 633 Execution count amp SyntaxError 1095 Uncaught TypeError Object has no method remove_and_cancell_tooltip 1075 We re ignoring prompt customizations 1086 Can t open qtconsole from outside source tree 1076 namespace changes broke foo bar syntax 1074 pprinting old style class objects fails TypeError tuple object is not callable 1063 Python utils test error due to missing unicodedata module 592 Bug in argument parsing for run 378 Windows path escape issues 1068 Notebook tab completion broken in Firefox 75 No tab completion after 103 customizable cpaste 324 Remove code in Python testing that is not being used 131 Global variables not seen by cprofile run 851 Python shell swallows exceptions in certain circumstances 882 ipython freezes at start if IPYTHONDIR is on an NFS mount 1057 Blocker Qt console broken after all magics menu became dynamic 1027 ipython does not like white space at end of file 1058 New bug Notebook asks for confirmation to leave even saved pages 1061 rep m
30. Ci Exception foo NameError Traceback most recent call last NameError name decorator is not defined a 5 Note Calling ipython qtconsole h will show all the style names that pygments can find on your system You can also pass the filename of a custom CSS stylesheet if you want to do your own coloring via the stylesheet argument The default LightBG stylesheet QPlainTextEdit QTextEdit background color white color black selection background color ccc error color red 238 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack in prompt color navy in prompt number font weight bold out prompt color darkred out prompt number font weight bold x inverted is used to highlight selected completion inverted background color black color white 4 5 5 Fonts The QtConsole has configurable via the ConsoleWidget To change these set the font_family or font_size traits of the ConsoleWidget For instance to use 9pt Anonymous Pro gt ipython qtconsole ConsoleWidget font_family Anonymous Pro ConsoleWidgat font_size 4 5 6 Process Management With the two process ZMQ model the frontend does not block input during execution This means that actions can be taken by the frontend while the Kernel is executing or even after it crashes The mo
31. Fix comment which breaks Emacs syntax highlighting fix Ygui magic Raise repr limit for strings to 80 characters from 30 don t use XDG path on OS X Unicode aware logger update zmqshell magics DOC Remove regression from qt console docs test_pr fallback on http if git protocol fail and SSL errors Fix some tests for Python 3 3 test for pygments before running qt tests 2 5 Issues closed in the 0 13 development cycle 95 IPython Documentation Release 1 2 1 An Afternoon Hack PR 1771 PR 1784 PR 1768 PR 1779 PR 1769 PR 1716 PR 1763 PR 1732 PR 1630 PR 1705 PR 1606 PR 1757 PR 1754 PR 1753 PR 1449 PR 1743 PR 1681 PR 1733 PR 1727 PR 1737 PR 1686 PR 1709 PR 1718 PR 1729 PR 1726 PR 1724 PR 1725 PR 1721 PR 1657 PR 1717 Make default value of interactivity passed to run_ast_nodes configurable restore loadpy to load Update parallel magics Tidy up error raising in magic decorators Allow cell mode timeit without setup code Fix for fake filenames in verbose traceback qtconsole fix append_plain_html gt append_html Refactoring of the magics system and implementation of cell magics Merge divergent Kernel implementations notebook Make pager resizable and remember size Share code for pycat and loadpy make pycat aware of URLs Open IPython notebook hyperlinks in a new window using target _blank Fix typo enconters gt encou
32. The ipython kernel subcommand has been added to allow starting a standalone kernel that can be used with various frontends You can then later connect a Qt console or a terminal console to this kernel by typing e g ipython qtconsole existing if it s the only one running or by passing explicitly the connection parameters printed by the kernel at startup 128 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack IPython File Edit View Kernel Magic Window Help kernel 1 X kernel0 slave 2 Python 2 7 2 default Oct 4 2011 20 06 09 Type copyright credits or license for more information IPython 0 12 An enhanced Interactive Python gt Introduction and overview of IPython s features quickref gt Quick reference help gt Python s own help system object gt Details about object use object for extra details sguiref gt A brief reference about the graphical user interface Welcome to pylab a matplotlib based Python environment backend module IPython zmq pylab backend inline For more information type help pylab In 1 imshow imread baboon png Out 1 lt matplotlib image AxesImage at 0x401c650 gt 100 Fig 2 4 The improved Qt console for Python now with tabs to control multiple kernels and full menu support 2 6 0 12 Series 129 IPython Documentation Release 1 2 1 An Af
33. __future__ environments skip ipc tests on Windows Fixing styling issues with CellToolbar fix KeyError creating ZMQStreams in notebook General cleanup of kernel manager code Initial Code to reduce parallel Client caching Exit code use type obj is cls as switch when canning Fix a breakpoint bug Remove outdated code from extensions autoreload P3K fix cookie parsing under Python 3 x duplicate import is removed In process kernel support take 3 WIP Metaui slideshow Chrome frame awareness Add version_request reply messaging protocol add px local for local execution Prefilter shouldn t touch execution_count UI For Metadata 40 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 2774 PR 2396 PR 2757 PR 2782 PR 2778 PR 2776 PR 2755 PR 2728 PR 2741 PR 2743 PR 2740 PR 2737 PR 2723 PR 2731 PR 2544 PR 2726 PR 2730 PR 2729 PR 2446 PR 2716 PR 2717 create a ipynbv3 json schema and a validator check for complete pyside presence before trying to import Allow the run magic with b to specify a file P3K fix Deprecation Warning under Python 3 x remove non functional View kill method can interactively defined classes Removing unused code in the notebook MappingKernelManager PR 2773 PR 2609 PR 2769 PR 2766 PR 2476 PR 2763 PR 2460 PR 2617 PR 2738 PR 2749 PR 2744
34. b while my_function a b becomes my_function a b e Extensible input syntax You can define filters that pre process user input to simplify input in special situations This allows for example pasting multi line code fragments which start with gt gt gt or such as those from other python sessions or the standard Python documentation e Flexible configuration system It uses a configuration file which allows permanent setting of all command line options module loading code and file execution The system allows recursive file inclusion so you can have a base file with defaults and layers which load other customizations for particular projects e Embeddable You can call Python as a python shell inside your own python programs This can be used both for debugging code or for providing interactive abilities to your programs with knowledge about the local namespaces very useful in debugging and data analysis situations e Easy debugger access You can set Python to call up an enhanced version of the Python debugger pdb every time there is an uncaught exception This drops you inside the code which triggered the exception with all the data live and it is possible to navigate the stack to rapidly isolate the source of a bug The run magic command with the d option can run any script under pdb s control automatically setting initial breakpoints for you This version of pdb has Python specific improve ments including tab comple
35. paths using raw Python strings r host share to make sure that the backslashes are properly escaped Starting the cluster profile Once a cluster profile has been configured starting an Python cluster using the profile is simple ipcluster start profile mycluster n 32 The n option tells ipcluster how many engines to start in this case 32 Stopping the cluster is as simple as typing Control C Using the HPC Job Manager fog When ipcluster start is run the first time ipcluster creates two XML job description files in the cluster directory e ipcontroller_job xml e ipengineset_job xml 5 10 Getting started with Windows HPC Server 2008 319 IPython Documentation Release 1 2 1 An Afternoon Hack Once these files have been created they can be imported into the HPC Job Manager application Then the controller and engines for that profile can be started using the HPC Job Manager directly without using ipcluster However anytime the cluster profile is re configured ipcluster start must be run again to regenerate the XML job description files The following screenshot shows what the HPC Job Manager interface looks like with a running Python cluster duster BLUE GNET COM HPC Job Manager File View Actions Options Help Q Back Forward Actions Job Management Active 2 All Jobs IPEngineSet Running GNET bgranger Highest 11 15 2009 11 30 14 PM Auto Auto PController Running GNET bgrange
36. 1518 Pretty printing exceptions is broken 811 Fixes for ipython unhandeled OSError exception on failure of os getcwdu 1517 Fix indentation bug in IPython lib pretty py 1519 BUG Include the name of the exception type in its pretty format 118 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 1525 1489 1401 1477 1514 1511 1506 1481 1241 1486 1485 1483 1383 1434 1268 1438 A hack for auto complete numpy recarray Fix zero copy push numpy arrays cannot be used with View apply in Python 3 fix dangling buffer in Python parallel util DOC Fix references to Python lib pretty instead of the old location Version comparison error 2 1 11 lt 2 1 4 gt True Fixing the Notebook scroll to help in visually comparing outputs BUG Improve placement of CallTipWidget When our debugger class is used standalone _oh key errors are thrown 676 Python embed from ipython crashes twice on exit 1496 1507 1508 1495 1505 1482 1501 1472 BUG LBYL when clearing the output history on shutdown python3 notebook TypeError unorderable types fix sorting profiles in clustermanager BUG Fix pretty printing for overzealous objects SQLite objects created in a thread can only be used in that same thread history documentation out of date dreload doesn t seem to exclude numpy more general fix for
37. 2038 add extra clear warning to shell doc 1265 please ship unminified js and css sources 2029 Ship unminified js 1920 Provide an easy way to override the Qt widget used by qtconsole 2007 Add custom_control and custom_page_control variables to override the Qt widgets used by qtconsole 2009 In magic help remove duplicate aliases 2033 ipython parallel pushing and pulling recarrays 2034 fix amp test push pull recarrays 2028 Reduce unhelpful information shown by pinfo 2 5 Issues closed in the 0 13 development cycle 103 IPython Documentation Release 1 2 1 An Afternoon Hack 1992 1885 2030 2024 2031 2023 2027 1901 2025 1986 1712 1183 2016 1806 2019 2020 2017 2021 1998 2022 1589 1996 2011 2014 2013 2005 2012 2004 2006 2010 2008 2003 Tab completion fails with many spaces in filename handle too old wx check wxPython version in inputhook Make interactive_usage a bit more rst friendly disable C C confirmation on Windows Unicode test failure on OS X match stdin encoding in frontend readline test Windows parallel test fails assert leaves 14 python processes alive Fix parallel test on WinXP wait for resource cleanup Line magic function R not found Rmagic test failure in ubuntu package daily build 0 12 testsuite failures BUG test runner fails in Windows if filenames contain spaces Alternate upload met
38. 2964 PR 2967 PR 2942 PR 2929 PR 2933 PR 2968 PR 2966 PR 2959 PR 2681 PR 2901 PR 2871 PR 2951 PR 2946 PR 2954 PR 2941 PR 2903 PR 2911 PR 2897 PR 2894 PR 2927 PR 2932 PR 2915 PR 2924 PR 2906 PR 2921 PR 2917 PR 2910 PR 2926 PR 2885 PR 2876 expanduser on file targets fix run all that gt this fix count when testing composite error output shows entire session history when only startsess is given Move CM IPython theme out of codemirror folder Cleanup cell insertion Minordocupdate fix notebook deletion Added assert msg to extract_hist_ranges Add command to trim the history database Don t enable pylab mode when matplotlib is not importable Fix inputhook_wx on osx truncate potentially long CompositeErrors use istype on lists tuples fix qtconsole history logic for end of line fix logic for append_javascript fix baseUrl Specify toggle value on cell line number display order in output area configurable Dont rely on BaseProjectUrl data in body tag Cm configurable next release will be 1 0 Simplify using notebook static files from external code added small config section to notebook docs page safe_run_module Silence SystemExit codes 0 and None Unpatch Monkey patch CM add menu item for undo delete cell Don t add logging handler if one already exists Respect DB_IP and DB_PORT in mongodb tests Don t die if stderr stdout do not suppo
39. 672 658 656 654 653 ipython with qtconsole crashes HTML Notebook Prompt can be deleted use argparse instead of kvloader for flags amp aliases Tornado version greater than 2 0 needed for firefox 6 Crash report when importing easter egg Ctrl Enter clears line in notebook ipengine fails on Windows with XXX lineno 355 opcode 0 Add python 3 tag to setup py non ascii in__doc__ string crashes qtconsole kernel when showing tooltip In Windows run fails to strip quotes from filename no completion in emacs by ipython ipython el Do not accept an ipython_dir that s not writeable segfault on mac os x RuntimeError Cannot change input buffer during execution in console_widget py Copy and paste keyboard shortcuts do not work in Qt Console on OS X PyZMQ s use of memoryviews breaks reconstruction of numpy arrays Turning off callout tips in qtconsole return kills Python Users should have intelligent autoindenting in the notebook Wireframe and implement a project dashboard page Wireframe and implement a notebook dashboard page Users should be able to use the notebook to import export a notebook to py or rst A user should be able to leave a kernel running in the notebook and reconnect Users should be able to save a notebook and then later reload it ipython qtconsole v0 11 setting c PythonWidget in_prompt gt gt gt crashes What happened to Exit Put the InteractiveShellApp section firs
40. 90 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack anatoly techtonik fawce mcelrath wilsaj We closed a total of 1115 issues 373 pull requests and 742 regular issues this is the full list generated with the script tools github_stats py Pull Requests 373 PR 1943 PR 1954 PR 2048 PR 2050 PR 2047 PR 2015 PR 2046 PR 1939 PR 2042 PR 2043 PR 2036 PR 1968 PR 2037 PR 2038 PR 2029 PR 2007 add screenshot and link into releasenotes update some example notebooks move _encode_binary to jsonutil encode_images only add quotes around xunit file on Windows disable auto scroll on mozilla Fixes for paste with special transformations Iptest unicode Namespaces increase auto scroll threshold to 100 lines move RemoteError import to top level alias_magic Proposal of icons for ipynb files remove ipython qtconsole gui script add extra clear warning to shell doc Ship unminified js Add custom_control and custom_page_control variables to override the Qt widgets used by qtconsole PR 2034 PR 2028 PR 2030 PR 2024 PR 2031 PR 2027 PR 2025 PR 2016 PR 2020 fix amp test push pull recarrays Reduce unhelpful information shown by pinfo check wxPython version in inputhook Make interactive_usage a bit more rst friendly disable C C confirmation on Windows match stdin encoding in frontend readline test Fix parallel t
41. A quick and easy method is to install it from a python session from IPython external mathjax import install_mathjax install_math jax 200 Chapter 3 Installation IPython Documentation Release 1 2 1 An Afternoon Hack If you need tighter configuration control you can download your own copy of MathJax from http www mathjax org download use the MathJax 2 0 link When you have the file stored locally in stall it with python m IPython external mathjax path to source mathjax MathJax v2 0 20 g07d69ac zip For unusual needs Python can tell you what directory it wants to find MathJax in python m IPython external mathjax d some other mathjax By default Mathjax will be installed in your ipython profile directory but you can make system wide install please refer to the documentation and helper function of IPython external mathjax 3 8 5 Browser Compatibility The Python notebook is officially supported on the following browers e Chrome 13 e Safari 5 e Firefox 6 The is mainly due to the notebook s usage of WebSockets and the flexible box model The following browsers are unsupported e Safari lt 5 e Firefox lt 6 e Chrome lt 13 e Opera any CSS issues but execution might work e Internet Explorer lt 10 The following specific combinations are known NOT to work e Safari IPython 0 12 tornado 2 2 0 e Safari with HTTPS connection to notebook and an untrusted certific
42. Dependencies for IPython parallel parallel computing 199 IPython Documentation Release 1 2 1 An Afternoon Hack 3 7 2 Qt Also with 0 11 a new GUI was added using the work in I Python kernel zmq which can be launched with ipython qtconsole The GUI is built on Qt and works with either PyQt which can be installed from the PyQt website or PySide from Nokia 3 7 3 pygments The syntax highlighting in ipython qtconsole is done with the pygments project which is easy_install able 3 8 Dependencies for the IPython HTML notebook The Python notebook is a notebook style web interface to Python and can be started with the command ipython notebook 3 8 1 pyzmq Like the TPython parallel and IPython frontend qt console packages the HTML note book requires ZeroMQ and PyZMQ 3 8 2 Tornado The Python notebook uses the Tornado project for its HTTP server Tornado 2 1 is required in order to support current versions of browsers due to an update to the websocket protocol 3 8 3 Jinja The Python notebook uses the Jinja templating tool to render HTML pages 3 8 4 MathJax The Python notebook uses the MathJax Javascript library for rendering LaTeX in web browsers Because MathJax is large we don t include it with IPython Normally Python will load MathJax from a CDN but if you have a slow network connection or want to use LaTeX without an internet connection at all you can install MathJax locally
43. Don t catch tab press when something selected Run All Above and Run All Below add GitHub uploads to release script Windows aware tests for shellglob Fix doctest_run_option_parser for Windows clear In prompt numbers again Clickable links Add encoding attribute to OutStream class ENH added StackExchange style MathJax filtering Fix traceback handling of SyntaxErrors without line numbers add missing qtconsole extras_require Add deprecation warnings for sympyprinting Make the ipengine monitor the ipcontroller heartbeat and die if the ipcontroller goes down PR 2479 PR 2474 use new _winapi instead of removed _subprocess fix bootstrap name conflicts 2 3 Issues closed in the 1 0 development cycle 43 IPython Documentation Release 1 2 1 An Afternoon Hack PR 2469 PR 2165 PR 2468 PR 2421 PR 2291 PR 2127 PR 2445 PR 2420 PR 2432 PR 2216 PR 1946 PR 2424 PR 2389 PR 2395 PR 1868 PR 2437 PR 2441 PR 2442 PR 2240 PR 2369 PR 2439 PR 2326 PR 2377 PR 2407 PR 2408 PR 2405 PR 2399 PR 2400 PR 2401 PR 2311 PR 2393 PR 2386 Treat __init__ pyc same as __init__ py in module_list Add g option to run to glob expand arguments Tell git to ignore __pycache__ directories Some notebook tweaks Remove old plugin system Ability to build toolbar in JS changes for ironpython Pass ipython_dir to __init__ method of TerminalInteractiv
44. Fixed minor typo causing AttributeError to be thrown Add unique option to history_request messaging protocol Allow shutdown when no engines are registered Define __ file _ when we edit a real file allow edit lt variable gt to work when interactively defined Reset readline delimiters after loading rmagic Better handling of __file__ when running scripts Fix for units argument Adds a res argument Unicode content crashes the pager console Tell Travis CI to test on Python 3 3 as well Don t show try paste message while using magics shift tab for tooltip Add note to cython Black Scholes example warning of missing erf BUG Octavemagic inline plots not working on Windows Fixed Following 2737 this error is now a name error Rmagic error message when moving an non existant variable from python to R diverse fixes for project url Rpush Look for variables in the local scope first Infinite loop when multiple debuggers have been attached Add qthelp docs creation added blockquote CSS Fix Read the doc build Again alternate 2267 Offline mathjax remove unexisting headings level One liner to fix debugger printing stack traces when lines of context are larger than source 2 3 Issues closed in the 1 0 development cycle 41 IPython Documentation Release 1 2 1 An Afternoon Hack PR 2713 PR 2690 PR 2691 PR 2699 PR 2700 PR 2692 PR 2685 PR 2686 PR 2604 PR 2689 PR 2683 PR
45. H He Set the arguments to be passed to ipcontroller note that remotely launched ipcontroller will not get the contents of the local ipcontroller_config py unless it resides on the xremote host in the location specified by the profile dir argument Se SR OF OSE c SSHControllerLauncher controller_args reuse ip profile dij Engines are specified in a dictionary by hostname and the number of engines to be run on that host c SSHEngineSetLauncher engines hostl example com 2 host2 example com oie host3 example com 1 profile dir home different location host4 example com 8 gt e The engines dict where the keys are the host we want to run engines on and the value is the number of engines to run on that host e on host3 the value is a tuple where the number of engines is first and the arguments to be passed to ipengine are the second element For engines without explicitly specified arguments the default arguments are set in a single location c SSHEngineSetLauncher engine_args profile dir path to profile_ssh Current limitations of the SSH mode of ipcluster are e Untested and unsupported on Windows Would require a working ssh on Windows Also we are using shell scripts to setup and execute commands on remote hosts Moving files with SSH SSH launchers will try to move connection files controlled by the to_send and
46. In 9 px print are you there Async parallel execution on engine s 0 2 Out 9 lt AsyncResult execute gt In 10 pxresult stdout 0 are you there stdout 2 are you there 5 4 2 Multiple Active Views The parallel magics are associated with a particular Direct View object You can change the active view by calling the activate method on any view In 11 even rce 2 In 12 even activate In 13 px print hi Async parallel execution on engine s 0 2 Out 13 lt AsyncResult execute gt In 14 even block True 5 4 Parallel Magic Commands 293 IPython Documentation Release 1 2 1 An Afternoon Hack In 15 px print hi Parallel execution on engine s 0 2 stdout 0 hi stdout 2 hi When activating a View you can also specify a suffix so that a whole different set of magics are associated with that view without replacing the existing ones restore the original DirecView to the base px magics In 16 rc activate Out 16 lt DirectView all gt In 17 even activate _even In 18 px print hi all Parallel execution on engine s all stdout 0 hi all stdout 1 hi all stdout 2 hi all stdout 3 hi al In 19 px_even print We aren t odd Parallel execution on engine s 0 2 stdout 0 We aren t odd stdout 2 We aren t odd This suffix is appli
47. PR 1224 string string Issues 742 PR 1306 PR 1301 PR 1289 PR 1288 PR 1294 PR 1283 PR 1264 PR 1284 PR 1213 PR 1267 PR 1222 PR 1257 PR 1253 PR 1262 PR 1229 PR 1256 PR 1246 PR 1211 PR 1187 PR 1207 PR 1129 PR 1199 PR 1218 PR 1217 PR 1175 PR 1196 PR 1190 Fix prun input parsing for escaped characters closes 1302 New Fix for issue 1202 based on current master Make autoreload extension work on Python 3 Don t ask for confirmation when stdin isn t available TaskScheduler hwm default to 1 instead of 0 HeartMonitor period should be an Integer Aceify a fix for GH 1269 BUG Minor typo in history_console_widget py add NoDB for non recording Hub allow Reference as callable in map apply use self kernel_manager_class in qtconsoleapp set auto_create flag for notebook apps Heartbeat no longer shares the app s Context Fix display of SyntaxError in Python 3 Dewijmoize Skip tests that require X when importing pylab results in RuntimeError serve local files in notebook dir edit text cells on double click instead of single click misc notebook connection file cleanup first heartbeat startup flush fix loadpy duplicating newlines Unified setup py Reduce IPython external Added q option to prun for suppression of the output along with editing the dochelp Added q option to prun for suppression of the output along with e
48. PR 2961 PR 2987 PR 2998 PR 3003 PR 2957 PyQt 4 10 use self _document self document propagate automagic change to shell don t give up on weird os names Fix example fix quarantine ipy_editors py Add info make target that was missing in old Sphinx A few small patches to image handling fix regular expression for detecting links in stdout restore default behavior for automatic cluster size fix ipython usage text fix DisplayMagics html docstring noted sub_channel being renamed to iopub_channel actually use IPKernelApp kernel_class Improve notebook js documentation add missing html magic check for SIGUSR1 before using it closes 3074 add width 100 to vbox for webkit FF consistency increase registration timeout fix DictDB default size limit on resume print server info again test double pyximport cast kernel cwd to bytes on Python 2 on Windows remove xml from notebook magic docstrings fix time format to international time format Fix test for Windows changed instances of outout to output in alt texts py3 workaround for reload in cythonmagic time magic shorten unnecessary output on windows fix local files examples in markdown fix css in output_area pre add include etc inputre to suggested inputre Refactor qt import logic Fixes 2955 2 3 Issues closed in the 1 0 development cycle 37 IPython Documentation Release 1 2 1 An Afternoon Hack PR 2994 PR 2983 PR
49. PR 3595 PR 3538 PR 3562 PR 3582 PR 3560 PR 3591 Added shorcuts to split cell merge cell above and merge cell below Added missing slash to print pdf call Drop patch for compatibility with pyreadline 1 5 Allow filename with extension in find_cmd in Windows Fix test for Python 3 on Windows Fix typo in docs use DEFAULT_STATIC_FILES_PATH in a test instead of package dir fix some unicode in zmqhandlers Set calling program to UNKNOWN when argv not in sys Set calling program to UNKNOWN when argv not in sys take 2 Use new entry point for python m Python passing cell to showInPager closes 3625 expand terminal color support raise UsageError for unsupported GUI backends Add magic function drun to run code in debugger a nicer error message when using pylab magic add extra_config_file updated mailmap Add examples for interactive use of MPI fix regular expression for ANSI escapes Corrected a typo in the format string for strftime the sphinx py transformer of nbconvert check for markdown no longer needed closes 3610 Simplify caching of modules with run notebook small things Fix duplicate completion in notebook parallel Improved logging for errors during BatchSystemLauncher stop Revert allow markdown in heading cells add Python start_ipython Allow custom nbconvert template loaders pandoc adjustments Remove max_msg_size Refer to Setuptools instead of Distribute 2 3 Issues closed
50. This load_ipython_extension function is called after your extension is imported and the cur rently active InteractiveShell instance is passed as the only argument You can do anything you want with Python at that point load_ipython_extension will be called again if you load or reload the extension again It is up to the extension author to add code to manage that Useful InteractiveShell methods include register_magic_function push to add variables to the user namespace and drop_by_id to remove variables on unloading You can put your extension modules anywhere you want as long as they can be imported by Python s standard import mechanism However to make it easy to write extensions you can also put your extensions in os path join ip ipython_dir extensions This directory is added to sys path automatically When your extension is ready for general use please add it to the extensions index 6 2 4 Extensions bundled with IPython autoreload Python extension to reload modules before executing user code autoreload reloads modules automatically before entering the execution of code typed at the Python prompt This makes for example the following workflow possible In 1 tload_ext autoreload In 2 Sautoreload 2 In 3 from foo import some_function In 4 some_function Out 4 42 In 5 open foo py in an editor and change some_function to return 43 In 6 some_function
51. To use MongoDB c HubFactory db_class IPython parallel controller mongodb MongoDB and SQLite c HubFactory db_class IPython parallel controller sqlitedb SQLiteDB You can use NoDB to disable the database altogether in case you don t need to reuse tasks or results and want to keep memory consumption under control c HubFactory db_class IPython parallel controller dictdb NoDB When using the proper databases you can actually allow for tasks to persist from one session to the next by specifying the MongoDB database or SQLite table in which tasks are to be stored The default is to use a table named for the Hub s Session which is a UUID and thus different every time To keep persistant task history in MongoDB c MongoDB database tasks and in SQLite c SQLiteDB table tasks Since MongoDB servers can be running remotely or configured to listen on a particular port you can specify any arguments you may need to the PyMongo Connection positional args to pymongo Connection c MongoDB connection_args keyword args to pymongo Connection c MongoDB connection_kwargs But sometimes you are moving lots of data around quickly and you don t need that information to be stored for later access even by other Clients to this same session For this case we have a dummy database which doesn t actually store anything This lets the Hub stay small in me
52. bug reports and ideas to this release Please do not hesitate in contacting us if we ve failed to acknowledge your contribution here In particular for this release we have contribution from the following people a mix of new and regular names in alphabetical order by first name 2 8 0 11 Series 163 IPython Documentation Release 1 2 1 An Afternoon Hack Aenugu Sai Kiran Reddy lt saikrn08 at gmail com gt andy wilson lt wilson andrew j github at gmail com gt Antonio Cuni lt antocuni gt Barry Wark lt barrywark at gmail com gt Beetoju Anuradha lt anu beethoju at gmail com gt Benjamin Ragan Kelley lt minrk at Mercury local gt Brad Reisfeld Brian E Granger lt ellisonbg at gmail com gt Christoph Gohlke lt cgohlke at uci edu gt Cody Precord dan milstein Darren Dale lt dsdale24 at gmail com gt Dav Clark lt davclark at berkeley edu gt David Warde Farley lt wardefar at iro umontreal ca gt epatters lt ejpatters at gmail com gt epatters lt epatters at caltech edu gt epatters lt epatters at enthought com gt Eric Firing lt efiring at hawaii edu gt Erik Tollerud lt erik tollerud at gmail com gt Evan Patterson lt epatters at enthought com gt Fernando Perez lt Fernando Perez at berkeley edu gt Gael Varoquaux lt gael varoquaux at normalesup org gt Gerardo lt muzgash at Muzpelheim gt Jason Grout lt jason grout at drake edu gt John Hunter lt jdh2358 at gmail com gt Jens Hedegaard N
53. files and all Python entry points extend it The Type and Instance traitlets now handle classes given as strings like foo bar Bar This is needed for forward declarations But this was implemented in a careful way so that string to class resolution is done at a single point when the parent HasTraitlets is instantiated IPython utils ipstruct has been refactored to be a subclass of dict It also now has full docstrings and doctests Created a Traits like implementation in IPython utils traitlets This is a pure Python lightweight version of a library that is similar to Enthought s Traits project but has no dependencies on Enthought s code We are using this for validation defaults and notification in our new component system Although it is not 100 API compatible with Enthought s Traits we plan on moving in 160 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack this direction so that eventually our implementation could be replaced by a yet to exist pure Python version of Enthought Traits Added a new module IPython 1ib inputhook to manage the integration with GUI event loops using PyOS_InputHook See the docstrings in this module or the main Python docs for details For users GUI event loop integration is now handled through the new gui magic command Type gui at an Python prompt for documentation For developers IPython lib inputhook provides a simple int
54. float In 7 view a Out 7 array 0 Tep Zay 3 el yy array 4 5 6 7 array 8 Sey Oxy They array 12 13 14 15 In 7 px totalsum psum a Parallel execution on engines 0 1 2 3 In 8 view totalsum Outils 120 0 120 0 120 0 120 0 Any Python code that makes calls to MPI can be used in this manner including compiled C C and Fortran libraries that have been exposed to Python 5 8 IPython s Task Database 5 8 1 Enabling a DB Backend The Python Hub can store all task requests and results in a database Currently supported backends are MongoDB SQLite and an in memory DictDB This database behavior is optional due to its potential Cost so you must enable one either at the command line gt ipcontroller dictb or mongodb or sqlitedb or in your ipcontroller_config py c HubFactory db_class DictDB c HubFactory db_class MongoDB c HubFactory db_class SQLiteDB 5 8 2 Using the Task Database The most common use case for this is clients requesting results for tasks they did not submit via In 1 re get_result task_id However since we have this DB backend we provide a direct query method in the Client for users who want deeper introspection into their task history The db_query method of the Client is modeled after 308 Chapter 5 Using IPython for parallel computing IPython D
55. get_dict error load_ext rmagic fails when legacy ipy_user_conf py is installed in ipython 0 13 1 OSX setp issue in ipython notebook with figure references nbconvert reveal to pdf conversion ignores styling prints only a single page Remove pyreadline workarounds we now require pyreadline gt 1 7 1 find_cmd test failure on Windows input in notebook doesn t work in Python 3 Deprecate as mathjax delimiter Pager does not open from button Miscellaneous small nbconvert feedback 256 color escapes support 52 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 3609 2934 3614 3610 3547 3602 3563 3599 2937 3479 3505 1537 3124 3464 2706 3587 3213 3580 3573 3544 3352 2971 pylab inline blows up for single process ipython Publish the Interactive MPI Demo Notebook ansi escapes broken in master ls color If you don t have markdown python setup py install says no pygments run modules clobber each other import_item fails when one tries to use DottedObjectName instead of a string Duplicate tab completions in the notebook Problems trying to run IPython on python3 without installing too long completion in notebook Write empty name for the notebooks nbconvert Failure in specifying user filter think a bit about namespaces Long multiline strings in Notebook run d message unclear IPython 0 13 1 ignor
56. has been moved into docs source as restructured text documents PDF and HTML documentation are being generated using Sphinx e New developer oriented documentation development guidelines and roadmap e Traditional ChangeLog has been changed to a more useful changes txt file that is organized by release and is meant to provide something more relevant for users Bug fixes e Created a proper MANIFEST in file to create source distributions e Fixed a bug in the Mult iEngine interface Previously multi engine actions were being collected with a DeferredList with fireononeerrback 1 This meant that methods were returning before all engines had given their results This was causing extremely odd bugs in certain cases To fix this problem we have 1 set fi reononeerrback 0 to make sure all results or exceptions are in before returning and 2 introduced a CompositeError exception that wraps all of the engine exceptions This is a huge change as it means that users will have to catch CompositeError rather than the actual exception Backwards incompatible changes All names have been renamed to conform to the lowercase_with_underscore convention This will require users to change references to all names like queueStatus to queue_status Previously methods like MultiEngineClient push and MultiEngineClient push used xargs and kwargs This was becoming a prob lem as we weren t able to introduce new keyword argument
57. ipython_config py profile default to inherit your default configuration as a starting point 6 1 4 Class based configuration inheritance There is another aspect of configuration where inheritance comes into play Sometimes your classes will have an inheritance hierarchy that you want to be reflected in the configuration system Here is a simple example from IPython config configurable import Configurable from IPython utils traitlets import Int Float Unicode Bool class Foo Configurable name Unicode u fooname config True value Float 100 0 config True class Bar Foo name Unicode u barname config True othervalue Int 0 config True Now we can create a configuration file to configure instances of Foo and Bar config file c get_config Foo name u bestname c Bar othervalue 10 Q 350 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack This class hierarchy and configuration file accomplishes the following e The default value for Foo name and Bar name will be bestname Because Bar is a Foo subclass it also picks up the configuration information for Foo e The default value for Foo value and Bar value will be 100 0 which is the value specified as the class default e The default value for Bar othervalue will be 10 as set in the configuration file Because Foo is the parent of
58. is accurate If the controller was started on a 278 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack remote machine you must copy that connection file to the client machine or enter its contents as arguments to the Client constructor If you have copied the json connector file from the controller In 2 re Client path to ipcontroller client json or to connect with a specific profile you have set up In 3 re Client profile mpi To make sure there are engines connected to the controller users can get a list of engine ids In Sle tesids ut Isli TO Ip 2 34 Here we see that there are four engines ready to do work for us For direct execution we will make use of a DirectView object which can be constructed via list access to the client In 4 dview rc use all engines See also For more information see the in depth explanation of Views 5 3 3 Quick and easy parallelism In many cases you simply want to apply a Python function to a sequence of objects but in parallel The client interface provides a simple way of accomplishing this using the DirectView s map method Parallel map Python s builtin map functions allows a function to be applied to a sequence element by element This type of code is typically trivial to parallelize In fact since IPython s interface is all about
59. lt http tools ietf org html rfc5246 gt OpenSSH lt http www openssh com gt Paramiko lt http www lag net paramiko gt HMAC lt http tools ietf org html rfc2104 html gt 427 IPython Documentation Release 1 2 1 An Afternoon Hack 428 Bibliography Python Module Index IPy IPy IPy thon ext thon ext tensions autoreload 356 tensions storemagic 358 thon ext 358 tensions sympyprinting 429 IPython Documentation Release 1 2 1 An Afternoon Hack 430 Python Module Index Index Symbols PATH 316 A AsyncResult built in class 338 E EDITOR 361 environment variable PATH 316 EDITOR 361 IPYTHON_DIR 87 351 IPYTHONDIR 87 351 PATH 2 PYTHONSTARTUP 423 G get AsyncResult method 338 IPython extensions autoreload module 356 IPython extensions storemagic module 358 IPython extensions sympyprinting module 358 IPYTHON_DIR 87 351 IPYTHONDIR 87 351 P PATH 2 PYTHONSTARTUP 423 R ready AsyncResult method 338 S successful AsyncResult method 338 W wait AsyncResult method 338 431
60. nbformat 3 nbformat_minor 0 worksheets eells cell_type markdown metadata source The simplest notebook Gell type code collapsed false input print Hello IPython language python metadata outputs f output_type stream stream stdout text Hello IPython n prompt_number 1 254 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack l metadata The corresponding Python script is i coding utf 8 lt nbformat gt 3 0 lt nbformat gt lt markdowncell gt The simplest notebook lt codecell gt print Hello IPython Note that indeed the output of the code cell which is present in the JSON container has been removed in the py script 4 8 Running a notebook server The Python notebook web application is based on a server client structure This server uses a fwo process kernel architecture based on ZeroMQ as well as Tornado for serving HTTP requests By default a notebook server runs on http 127 0 0 1 8888 and is accessible only from localhost This document describes how you can secure a notebook server and how to run it on a public interface 4 8 1 Notebook security You can protect your notebook server with a simple single password by setting the NotebookApp password configurable Y
61. ns Message type dat a_pub 398 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack content the keys of the data dict after it has been unserialized keys a b the namespace dict will be serialized in the message buffers which will have a length of at least one buffers pdict The interpretation of a sequence of data_pub messages for a given parent request should be to update a single namespace with subsequent results Note No frontends directly handle data_pub messages at this time It is currently only used by the client engines in IPython parallel where engines may publish data to the Client of which the Client can then publish representations via display_data to various frontends Python inputs These messages are the re broadcast of the execute_request Message type pyin content code Str Source code to b xecuted one or more lines The counter for this execution is also provided so that clients can display it since IPython automatically creates variables called _iN for input prompt In N execution_count int Python outputs When Python produces output from code that has been compiled in with the single flag to compile any expression that produces a value such as 1 1 is passed to sys displayhook which is a callable that can do with this
62. precision for controlling float and numpy pretty printing Python applications initiate logging so any object can gain access to a the logger of the currently running Application with 158 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack B MpiDemo Microsoft Visual Studio ale File Edit View Refactor Project Build Debug Team Data Tools Architecture Test Analyze Window Help igd a Gda s aaly e f gt Debug AddClass Fh se el New Workltem 3 Jy sli C h h ae fE anre eP AS reee s MpiDemo LUGE x Python 2 7 Interactive X Solution Explorer vax wae In 1 from IPython parallel import Client a od Solution MpiDemo 1 pr This example computes PI to certain precision using 2 re Client 4 amp MpiDemo 4 processors and a monte carlo simulation 3 re ids Search Path 3 2 3 O Program py 4 dview rc use all engines import random 5 serial_result map lambda x x 10 range 32 from mpi4py import MPI 7 6 parallel_result dview map_sync lambda x comm MPI COMM_WORLD x 1 range 32 import numpy as np In 7 serial_result parallel_result True def computePi nsamples In 8 parallel_result rank size comm Get_rank comm Get_size e oldpi pi mypi 0 0 0 0 0 1 1024 done False 59049 while not done 1048576 ezi inside 9765625 for i in xrange nsamples 60466176 x random random 28
63. subplot 122 sombrero 80 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack R magics extension rmagic This extension provides several magics that support calling code written in the R language for statistical data analysis You can execute single lines or whole blocks of R code capture both output and figures inline just like matplotlib plots and have variables automatically converted between the two languages To use this extension you must have R installed as well as the rpy2 package that bridges Python and R The examples directory in the source distribution ships with a full notebook demonstrating these capabilities 2 4 0 13 Series 81 IPython Documentation Release 1 2 1 An Afternoon Hack Py rmagic_extension x G2 127 0 0 1 3889 7641d634 8113 4ed4 8dc0 264444d3574d PAE ORR S IP y Notebook rmagic_extension Last saved Jun 29 11 20 PM File Edit View Insert Cell Kernel Help B x 6 Tt g gt Code v We can redo the above calculations in one ipython cell We might also want to add some output such as a summary from R or perhaps the standard plotting diagnostics of the Im In 16 R i X Y o XYcoef XYlm lm Y X XYcoef coef XYlm print summary XYlm par mfrow c 2 2 plot xYlm Call lm formula Y X Residuals 2 2 3 4 0 2 0 9 1 0 6 1 5 0 2 Coefficients Estimate Std Error t value Pr gt
64. t Intercept 3 2000 0 6164 5 191 0 0139 x 0 9000 0 2517 3 576 0 0374 Signif coles 0 42 6 601 01 1 0 05S 6 1 7 12 Residual standard error 0 7958 on 3 degrees of freedom Multiple R squared 0 81 Adjusted R squared 0 7467 F statistic 12 79 on 1 and 3 DF p value 0 03739 Residuals vs Fitted Normal Q Q 1 0 10 05 00 05 Standardized residuals 10 05 00 05 10 Theoretical Quantiles Scale Location Residuals vs Leverage tandardized residuals Standardized residuals 0 0 01 02 03 04 05 06 Leverage 82 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Tab completer improvements Useful tab completion based on live inspection of objects is one of the most popular features of Python To make this process even more user friendly the completers of both the Qt console and the Notebook have been reworked The Qt console comes with a new ncurses like tab completer activated by default which lets you cycle through the available completions by pressing tab or select a completion with the arrow keys PR 1851 In 2 import numpy random as random In 3 numpy random lt tab gt beta logistic power standard_exponential binomial lognormal rand standard_gamma bytes logseries randint standard_normal chisquare mtrand randn standard_t dirichlet multinomial random test exponential multivariate_normal random_integers triangular f negat
65. 0 1 0 0 3 0 03 0 1 Out 7 10 327861752731728 In 8 timeit black_scholes 100 0 100 0 1 0 0 3 0 03 8 0 1 1000000 loops best of 3 821 ns per loop Octave magics extension ocfavemagic This extension provides several magics that support calling code written in the Octave language for numerical computing You can execute single lines or whole blocks of Octave code capture both output and figures inline just like matplotlib plots and have variables automatically converted between the two languages To use this extension you must have Octave installed as well as the oct2py package The examples directory in the source distribution ships with a full notebook demonstrating these capabilities 2 4 0 13 Series 79 IPython Documentation Release 1 2 1 An Afternoon Hack IPy octavemagic_extension x a a 127 0 0 1 8889 34f7dfba fb06 44ab af09 add90ad3ccd3 Zoa IP y Notebook octavemagic_extension Last saved Jun 29 11 35 PM File Edit View Insert Cell Kernel Help 8 x 6 tjd a gt Heading1 Y v 7 v v In 24 octave s 500 500 butterworth filter order 2 cutoff pi 2 radians b 0 292893218813452 0 585786437626905 292893218813452 a 1 6 171572875253810 freqz b a 32 hase degrees 0 0 10 20 30 40 50 60 70 80 9 Pramono octave s 600 200 f png subplot 121 x y meshgrid 0 0 1 3 r sin x 80 5 2 cos y 5 2 surf x y r
66. 0 development cycle 33 IPython Documentation Release 1 2 1 An Afternoon Hack PR 3276 PR 3301 PR 3282 PR 3284 PR 3281 PR 3275 PR 3239 PR 3290 PR 3142 PR 3066 PR 3269 PR 3267 PR 3101 PR 3264 PR 3263 PR 3258 PR 3254 PR 3240 PR 3158 PR 3252 PR 3251 PR 3250 PR 3249 PR 3235 PR 3238 PR 3242 PR 3219 PR 3190 PR 3231 PR 3233 PR 3234 PR 3232 Fix python ABS PATH TO ipython py fails allow python3 tests without python installed allow view map to work with a few more things remove ipython py entry point fix ignored IOPub messages with no parent improve submodule messages git hooks Allow x icon and esc key to close pager in notebook Improved heartbeat controller to engine monitoring for long running tasks Better error message when CWD doesn t exist on startup Add support for relative import to run m fixes 2727 protect highlight js against unknown languages add missing return use marked highlight js instead of pagedown and prettify use https url for submodule fix set_last_checkpoint when no checkpoint Fix submodule location in setup py fix a few URLs from previous PR remove js components from the repo IPEP 15 autosave the notebook move images out of _ static folder into _images Fix for cell magics in Qt console Added a simple __html__ method to the HTML class remove copy of sphinx inheritance_diagram py Remove the unus
67. 1363 PR 1187 PR 1190 PR 1196 PR 1206 PR 1207 PR 1218 PR 1222 PR 1229 PR 1246 PR 1253 PR 1257 PR 1262 PR 1283 PR 1284 PR 1289 PR 1306 PR 1312 PR 1318 PR 1341 PR 1353 core completer Clean up excessive and unused code misc notebook connection file cleanup first heartbeat startup flush Fix link to Chris Fonnesbeck blog post about 0 11 highlights docs looks like a file path might have been accidentally pasted in the middle of a word don t preserve fixConsole output in json fix loadpy duplicating newlines BUG Minor typo in history_console_widget py Added q option to prun for suppression of the output along with editing the dochelp allow Reference as callable in map apply Fix display of SyntaxError in Python 3 Skip tests that require X when importing pylab results in RuntimeError set auto_create flag for notebook apps use self kernel_manager_class in qtconsoleapp Heartbeat no longer shares the app s Context HeartMonitor period should be an Integer a fix for GH 1269 Make autoreload extension work on Python 3 Fix Yprun input parsing for escaped characters closes 1302 minor heartbeat tweaks make Ctrl D in qtconsole act same as in terminal ready to merge Don t attempt to tokenize binary files for tracebacks Save notebook as script using unicode file handle Fix some minor color style config issues in the qtconsole 134 Chapter 2 What s
68. 1560 __all___ feature improvement to dir2 and tests for both Custom namespace for run calling abort on AsyncMapResult results in traceback add sugar methods properties to AsyncResult Fix pretty printing dispatch Discussion some potential Qt console refactoring Use LaTeX to print various built in types with the SymPy printing extension re enter kernel eventloop after catching SIGINT rename plaintext cell gt raw cell Yonotebook fails in qtconsole trailing newline not preserved in splitline ipynb Fix Ynotebook magic etc nbformat unicode tests and fixes Gtk3 integration with ipython works Examples syntax avoid errors installing on Python 3 Find encoding for Python files Fix writing git commit ID to a file on build with Python 3 shallow copy DictDB query results various pyflakes issues small changes in response to pyflakes pass Don t build sphinx docs for sdists unhide git_commit_info ini store git commit hash in utils _sysinfo instead of hidden data file attempt to suppress exceptions in channel threads at shutdown unhide git_commit_info ini update tools github_stats py to use GitHub API v3 clear_output improvements Ipython testing documentation still mentions twisted and trial remove obsolete discussion of Twisted trial from testing docs 2 5 Issues closed in the 0 13 development cycle 117 IPython Documentation Release 1 2 1 An Afternoon Hack 1561 Qtconsole nonstandard a and b 1569 BUG
69. 2 See which files have changed with git status see git status You ll see a listing like this one On branch ny new feature Changed but not updated use git add lt file gt to update what will be committed use git checkout lt file gt to discard changes in working directo modified README ry 378 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack Untracked files use git add lt file gt to include in what will be committed INSTALL no changes added to commit use git add and or git commit a 3 Check what the actual changes are with git diff git diff 4 Add any new files to version control git add new_file_name see git add 5 To commit all modified files into the local copy of your repo do git commit am A commit message Note the am options to commit The m flag just signals that you re going to type a message on the command line The a flag you can just take on faith or see why the a flag See also the git commit manual page 6 To push the changes up to your forked repo on github doa git push see git push Asking for code review 1 Go to your repo URL e g http github com your user name ipython 2 Click on the Branch list button Source Commits Network 1 2 Fork Queue Switch Branches 1
70. Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 2545 2080 2541 2508 2289 2487 2521 2535 2523 2514 2220 2512 2496 1453 2481 2488 2482 2471 2374 2316 2457 1575 2097 2435 2429 2422 2426 pypi win32 installer creates 64bit executibles Event loop issues with IPython 0 12 and PyQt4 QDialog exec_ and more Allow python m IPython subplots_adjust does not work correctly in ipython notebook Incorrect mathjax rendering of certain arrays of equations Selecting and indenting more fine grained run controls such as run from here and run until here Funny bounding box when plot with text History not working Issue with zooming in qtconsole No sys stdout encoding in kernel based Python ERROR Internal Python error in the inspect module Function passwd does not work in QtConsole make engines reconnect die when controller was restarted ipython notebook clicking in a code cell s output moves the screen to the top of the code cell Undesired plot outputs in Notebook inline mode ipython notebook download may not get the latest notebook _subprocess module removed in Python 3 3 Issues with man pages parallel Client __init__ should take cluster_id kwarg Can a R library wrapper be created with Rmagic Fallback frontend for console when connecting pylab inInie enabled kernel Do not crash if histor
71. Copyright c 2006 Microsoft Corporation All rights reserved Z gt ipython Python 2 7 2 default Jun 12 2011 15 08 59 MSC v 1500 32 bit Intel Type copyright credits or license for more information IPython 0 12 dev An enhanced Interactive Python gt Introduction and overview of IPython s features squickref gt Quick reference help gt Python s own help system object gt Details about object use object for extra details In 1 5 10 4 Starting an IPython cluster To use Python s parallel computing capabilities you will need to start an Python cluster An Python cluster consists of one controller and multiple engines IPython controller The Python controller manages the engines and acts as a gateway between the engines and the client which runs in the user s interactive Python session The controller is started using the ipcontroller command IPython engine Python engines run a user s Python code in parallel on the compute nodes Engines are starting using the ipengine command Once these processes are started a user can run Python code interactively and in parallel on the engines from within the Python shell using an appropriate client This includes the ability to interact with plot and visualize data from the engines Python has a command line program called ipcluster that automates all aspects of starting the controller and engines on the compute nodes ipcluster has
72. Feser Jake Vanderplas e jakobgager James Booth Jan Schulz Jason Grout Jeff Knisley Jens Hedegaard Nielsen jeremiahbuddha Jerry Fowler Jessica B Hamrick Jez Ng John Zwinck Jonathan Frederic Jonathan Taylor Joon Ro Joseph Lansdowne Juergen Hasch Julian Taylor Jussi Sainio 2 3 Issues closed in the 1 0 development cycle 21 IPython Documentation Release 1 2 1 An Afternoon Hack J rgen Stenarson kevin klonuo Konrad Hinsen Kyle Kelley Lars Solberg Lessandro Mariano Mark Sienkiewicz at STScI Martijn Vermaat Martin Spacek Matthias Bussonnier Maxim Grechkin Maximilian Albert MercuryRising Michael Droettboom Michael Shuffett Micha G rny Mikhail Korobov mr Shu Nathan Goldbaum ocefpaf Ohad Ravid Olivier Grisel Olivier Verdier Owen Healy Pankaj Pandey Paul Ivanov Pawel Jasinski Pietro Berkes Piti Ongmongkolkul Puneeth Chaganti Rich Wareham 22 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Richard Everson Rick Lupton Rob Young Robert Kern Robert Marchman Robert McGibbon Rui Pereira Rustam Safin Ryan May s8weber Samuel Ainsworth Sean Vig Siyu Zhang Skylar Saveland slojo404 smithy 1 Stefan Karpinski Stefan van der Walt Steven Silvester Takafumi Arakaki Takeshi Kanmae temulcahy teegaar Thomas Kluyver Thomas Robitaille Thomas Spura Thomas Wei
73. First start the controller ipcontroller Next start however many instances of the engine you want using repeatedly the command ipengine The engines should start and automatically connect to the controller using the JSON files in IPYTHONDIR profile_default security You are now ready to use the controller and engines from IPython Warning The order of the above operations may be important You must start the controller before the engines unless you are reusing connection information via reuse in which case ordering is not important Note On some platforms OS X to put the controller and engine into the background you may need to give these commands in the form ipcontroller amp and ipengine amp with the parentheses for them to work properly Starting the controller and engines on different hosts When the controller and engines are running on different hosts things are slightly more complicated but the underlying ideas are the same 1 Start the controller on a host using ipcontroller The controller must be instructed to listen on an interface visible to the engine machines via the ip command line argument or HubFactory ip in ipcontroller_config py ipcontroller ip 192 168 1 16 in ipcontroller_config py HubFactory ip 192 168 1 16 2 Copy ipcontroller engine json from IPYTHONDIR profile_ lt name gt security on the controller s host to the
74. GNU readline library so they won t work if your Python installation lacks readline support We will first describe the default behavior Python uses and then how to change it to suit your preferences 4 3 IPython reference 213 IPython Documentation Release 1 2 1 An Afternoon Hack Command line completion At any time hitting TAB will complete any available python commands or variable names and show you a list of the possible completions if there s no unambiguous one It will also complete filenames in the current directory if no python names match what you ve typed so far Search command history Python provides two ways for searching through previous input and thus reduce the need for repetitive typing 1 Start typing and then use Ctrl p previous up and Ctrl n next down to search through only the history items that match what you ve typed so far If you use Ctrl p Ctrl n at a blank prompt they just behave like normal arrow keys 2 Hit Ctrl r opens a search prompt Begin typing and the system searches your history for lines that contain what you ve typed so far completing as much as it can Persistent command history across sessions Python will save your input history when it leaves and reload it next time you restart it By default the history file is named IPYTHONDIR profile_ lt name gt history sqlite This allows you to keep separate his tories related to various tasks commands related to numeric
75. InteractiveShel11 have been organized into sections to make it easier to turn more sections of functionality into components e The embedded shell has been refactored into a truly standalone subclass of InteractiveShell 2 8 0 11 Series 159 IPython Documentation Release 1 2 1 An Afternoon Hack called InteractiveShellEmbed All embedding logic has been taken out of the base class and put into the embedded subclass Added methods of InteractiveShel11 to help it cleanup after itself The cleanup method controls this We couldn t do this in ___de1___ because we have cycles in our object graph that prevent it from being called Created a new module IPython utils importstring for resolving strings like foo bar Bar to the actual class Completely refactored the IPython core prefilter module into Configurable sub classes Added a new layer into the prefilter system called transformations that all new prefilter logic should use rather than the older checker handler approach Aliases are now components IPython core alias New top level embed function that can be called to embed Python at any place in user s code On the first call it will create an InteractiveShellEmbed instance and call it In later calls it just calls the previously created Interact iveShellEmbed Created a configuration system IPython config configurable that is based on IPython utils traitlets Configurables are arrang
76. Python 2 6 6 2 3 3 Issues closed in 1 0 GitHub stats for 2012 06 30 2013 08 08 since 0 13 These lists are automatically generated and may be incomplete or contain duplicates The following 155 authors contributed 4258 commits Aaron Meurer Adam Davis Ahmet Bakan Alberto Valverde Allen Riddell Anders Hovm ller Andrea Bedini Andrew Spiers Andrew Vandever Anthony Scopatz Anton Akhmerov Anton I Sipos Antony Lee Aron Ahmadia Benedikt Sauer Benjamin Jones Benjamin Ragan Kelley Benjie Chen Boris de Laage 2 3 Issues closed in the 1 0 development cycle 19 IPython Documentation Release 1 2 1 An Afternoon Hack Brad Reisfeld Bradley M Froehle Brian E Granger Cameron Bates Cavendish McKay chapmanb Chris Beaumont Chris Laumann Christoph Gohlke codebraker codespaced Corran Webster DamianHeard Damian Avila Dan Kilman Dan McDougall Danny Staple David Hirschfeld David P Sanders David Warde Farley David Wolever David Wyde debjan Diane Trout dkua Dominik Dabrowski Donald Curtis Drazen Lu anin drevicko Eric O LEBIGOT Erik M Bray Erik Tollerud 20 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Eugene Van den Bulke Evan Patterson Fernando Perez Francesco Montesano Frank Murphy Greg Caporaso Guy Haskin Fernald guziy Hans Meine e Harry Moreno henryili Ivan Djokic Jack
77. Python Documentation Release 1 2 1 An Afternoon Hack The IPython Development Team September 25 2015 Contents 1 Introduction EE OVERVIEW na nesa ab os OR oe eh a ee ele bg he a e 1 2 Enhanced interactive Python shell sorce w ata u ee we ee 1 3 Decoupled two process model o o s sosom eca ace 0200002 eee eee 1 4 Interactive parallel computing sis a wees ai ed iaon aia ee a a 2 What s new in IPython 21 Development Version 4 4604 206 4ou bese ee dda Sead ead a 2 2 MOSES 8 0 Bsus Ga Gos oe eae Ge AES dea ed he oS Hal es 2 3 Issues closed in the 1 0developmentcycle DA WOM SEES o ar eee howe eter pee okt a pea anor eras He eee os 2 5 Issues closed in the 0 13 development cycle 26 ON 2 Senes asan gw eels a Se A ee A a a a E 2 7 Issues closed in the 0 12 development cycle 287 OMI SEMES fist ah es ee we Ee ia ew oe we 2 9 Issues closed in the 0 11 development cycle 0 2 ANO AO WOSETICS yoga 5 we doe Ge ie Back de RRS Ba He a Gow aa A DAM MOOSEMCS yk ree e e a SE A eG Jee Oe Be ae ess Bae As Dodd OSS EHESS ce wrk ole gee a ete gm a 0 ee ENS Oe ele eee a ew 3 Installation Sal MQUICKStANG sede its HA Pa Sees Beh ee he ithe wee Ai ie se i en Ee a Ba ONEVIEW rerin doe dow Se a Oe Pee ou eG oi Oe AG doe amp 3 3 Installing Python itself 24 44 440 2a 24 Ba 4a Pe ew Roe ea geo 34 Basicoptional dependencies ss sens hn baw ee
78. Release 1 2 1 An Afternoon Hack 3 2 Overview This document describes in detail the steps required to install Python and its various optional dependencies For a few quick ways to get started with package managers or full Python distributions see the install page of the Python website Python is organized into a number of subpackages each of which has its own dependencies All of the subpackages come with Python so you don t need to download and install them separately However to use a given subpackage you will need to install all of its dependencies Please let us know if you have problems installing Python or any of its dependencies IPython and most dependencies can be installed via easy_install provided by the setuptools package or pip In many scenarios this is the most simplest method of installing Python packages More information about setuptools can be found on its PyPI page Note On Windows IPython requires setuptools We hope to change this in the future but for now on Windows you must install setuptool1s to use Python More general information about installing Python packages can be found in Python s documentation 3 3 Installing IPython itself Given a properly built Python the basic interactive Python shell will work with no external dependencies However some Python distributions particularly on Windows and OS X don t come with a working readline module The Python shell will work wit
79. Work around lack of os kill in win32 win32 iptest Use subprocess Popen instead of os system Pylab switch Add an API for registering magic aliases ipdb pdef pdoc pinfo magics all broken Ensure consistent indentation in magic add size limiting to the DictDB backend Fix Python lib latextools for Python 3 removed references to hSpy dependence in octave magic documentation Include the kernel object in the event object passed to kernel events added test for store fixed storemagic Use breqn sty in dvipng backend if possible handle undefined param in notebooklist fix 1814 set __file__ when running ipy files Add a metadata attribute to messages simplify Python parallel connections and enable Controller Resume add javascript o svg and latex display magics different images in 00_notebook tour prun Restore stats stream after running print_stream show message on notebook list if server is unreachable 2 3 Issues closed in the 1 0 development cycle 47 IPython Documentation Release 1 2 1 An Afternoon Hack PR 2176 PR 2152 PR 2177 PR 2090 PR 2160 PR 2168 PR 2170 PR 2096 PR 2163 PR 2158 PR 2089 PR 2142 PR 2140 PR 2131 PR 2117 PR 2133 PR 2134 PR 2126 PR 2104 PR 2074 PR 2123 PR 2100 PR 2128 PR 2073 PR 1993 PR 2086 PR 2110 PR 2101 PR 2102 PR 2088 PR 2064 PR 2076 fix git mpr qtconsole Namespace not empty at start
80. a C compiler for this to work The examples directory in the source distribution ships with a full notebook demonstrating these capabilities 78 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack The cython magic Probably the most important magic is the cython magic This is similar to the cython_pyximport magic but doesn t require you to specify a module name Instead the cython magic uses manages everything using temporary files in the cython magic directory All of the symbols in the Cython module are imported automatically by the magic Here is a simple example of a Black Scholes options pricing algorithm written in Cython In 6 cython cimport cython from libc math cimport exp sqrt pow log erf cython cdivision True cdef double std_norm_cdf double x nogil return 0 5 1 erf x sqrt 2 0 cython cdivision True def black_scholes double s double k double t double v double rf double div double cp Price an option using the Black Scholes model initial stock price strike price expiration time v volatility rf risk free rate div dividend cp 1 1 for call put sae Arv cdef double d1 d2 optprice with nogil d1 log s k rf div 0 5 pow v 2 t v sq rt t d2 d1 v sqrt t optprice cp s exp div t std_norm_cdf cp d1 cp k exp rf t std_norm_cdf cp d2 return optprice In 7 black_scholes 100 0 100
81. about it is our Python 3 wiki page Unicode Entering non ascii characters in unicode literals u now works properly on all platforms However entering these in byte string literals will not work as expected on Windows or any platform where the terminal encoding is not UTF 8 as it typically is for Linux amp Mac OS X You can use escape sequences xe9 x82 to get bytes above 128 or use unicode literals and encode them This is a limitation of Python 2 which we cannot easily work around Integration with Microsoft Visual Studio Python can be used as the interactive shell in the Python plugin for Microsoft Visual Studio as seen here The Microsoft team developing this currently has a release candidate out using Python 0 11 We will continue to collaborate with them to ensure that as they approach their final release date the integration with Python remains smooth We d like to thank Dino Viehland and Shahrokh Mortazavi for the work they have done towards this feature as well as Wenming Ye for his support of our WinHPC capabilities Additional new features Added Bytes traitlet removing Str All string traitlets should either be Unicode if a real string or Bytes if a C string This removes ambiguity and helps the Python 3 transition New magic loadpy loads a python file from disk or web URL into the current input buffer New magic pastebin for sharing code via the Lodge it pastebin New magic
82. about the displayed object This is used to set size etc for retina graphics To enable retina matplotlib figures simply set InlineBackend figure_format retina for 2x PNG figures in your Python config file or via the config magic 10 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Add display FileLink and FileLinks for quickly displaying HTML links to local files Cells have metadata which can be edited via cell toolbars This metadata can be used by external code e g reveal js or exporters when examining the notebook Fix an issue parsing LaTeX in markdown cells which required users to type instead of Notebook templates are rendered with Jinja instead of Tornado e ol o9 file has been renamed b writefile file is deprecated ANSI and VT100 color parsing has been improved in both performance and supported values The static files path can be found as IPython html DEFAULT_STATIC_FILES_PATH which may be changed by package managers IPython s CSS is installed in static css style min css all style including bootstrap and static css ipython min css which only has IPython s own CSS The latter file should be useful for embedding Python notebooks in other pages blogs etc The Print View has been removed Users are encouraged to test ipython nbconvert to generate a static view Javascript Components The javascript components u
83. all prompt numbers when doing a Clear all output This reduces the amount of noise in commit to commit diffs that would otherwise show the highly variable prompt number changes PR 1621 The notebook server now requires two consecutive Ct r1 C within 5 seconds or an interactive confir mation to terminate operation This makes it less likely that you will accidentally kill a long running server by typing Ctrl C in the wrong terminal PR 1609 Using Ct r1 S or Cmd S on a Mac actually saves the notebook rather than providing the fairly useless browser html save dialog PR 1334 Allow accessing local files from the notebook in urls by serving any local file as the url files lt relativepath gt This makes it possible to for example embed local images in a note book PR 1211 2 4 0 13 Series 75 IPython Documentation Release 1 2 1 An Afternoon Hack IPy IPython Dashboard xj IPy 00_notebook_tour IPI y Notebook 00_notebook_tour Last saved Jun 13 10 14 AM File Edit View insert Cell Kernel Help 8 x t 4 7 gt Code Rich displays include anyting a browser can show Note that we have an actual protocol for this see the display_protocol notebook for further details Images In 1 from IPython core display import Image Image filename source _static logo png Out 1 Image self data None url None filename None format u png embed None x Create a di
84. are using the notebook qtconsole or parallel code Python stores connection information in small JSON files in the active profile s security directory This directory is made private so only you can see the files inside If you need to move connection files around to other computers this is where they will be If you want your code to be able to open security files by name we have a convenience function IPython utils path get_security_file which will return the absolute path to a security file from its filename and optionally profile name 352 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack Startup Files If you want some code to be run at the beginning of every Python session with a particular profile the easiest way is to add Python py or Python ipy scripts to your lt profile gt startup directory Files in this directory will always be executed as soon as the Python shell is constructed and before any other code or scripts you have specified If you have multiple files in the startup directory they will be run in lexicographical order so you can control the ordering by adding a 00 prefix 6 1 7 Command line arguments Python exposes all configurable options on the command line The command line arguments are generated from the Configurable traits of the classes associated with a given Application Configuring Python from the command line may l
85. be able to see any output of the kernel They will NOT be able to issue shell commands as you due to message signatures which are enabled by default as of Python 0 12 Warning If you disable message signatures then any user with access to the ports your kernel is listening on can issue arbitrary code as you DO NOT disable message signatures unless you have a lot of trust in your environment The one security feature Python does provide is protection from unauthorized execution Python s mes saging system will sign messages with HMAC digests using a shared key The key is never sent over the network it is only used to generate a unique hash for each message based on its content When Python receives a message it will check that the digest matches and discard the message You can use any file that only you have access to to generate this key but the default is just to generate a new UUID You can generate a random private key with generate 1024b of random data and store in a file only you can read assumes IPYTHONDIR is defined otherwise use your IPython directory gt python c import os print os urandom 128 encode base64 gt SIPYTHONDIR sessionkey gt chmod 600 SIPYTHONDIR sessionkey The contents of this file will be stored in the JSON connection file so that file contains everything you need to connect to and use a kernel To use this generated key simply specify the Session keyfile confi
86. call last gt 1 1 0 ZeroDivisionError integer division or modulo by zero 3 execute ZeroDivisionError Traceback most recent call last gt 1 1 0 ZeroDivisionError integer division or modulo by zero In 82 debug gt site packages IPython parallel client asyncresult py 125 get 124 else e gt 125 raise self _exception 126 else Here self _exception is the CompositeError instance ipdb gt self _exception ipdb gt e CompositeError 4 we can tab complete on to s available methods ipdb gt e lt TAB gt e args e message e traceback e elist e msg e ename e print_traceback e engine_info e raise_exception e evalue e render_traceback We can then display the individual tracebacks if we want ipdb gt e print_traceback 1 l execute ZeroDivisionError Traceback most recent call last gt 1 1 0 ZeroDivisionError integer division or modulo by zero Since you might have 100 engines you probably don t want to see 100 tracebacks for a simple NameEr ror because of a typo For this reason CompositeError truncates the list of exceptions it will print to CompositeError tb_limit default is five You can change this limit to suit your needs with In 20 from IPython parallel import CompositeError In 21 CompositeError tb_limit 1 In 22 px a b O execute NameError Traceback most recent call last
87. code cell notebook name The name of the notebook document is displayed at the top of the page next to the IP y Notebook logo This name reflects the name of the ipynb notebook document file Clicking on the notebook name brings up a dialog which allows you to rename it Thus renaming a notebook from UntitledO to My first notebook in the browser renames the Untitled0 ipynb file toMy first notebook ipynb menu bar The menu bar presents different options that may be used to manipulate the way the notebook functions toolbar The tool bar gives a quick way of performing the most used operations within the notebook by clicking on an icon code cell the default type of cell read on for an explanation of cells 4 6 4 Structure of a notebook document The notebook consists of a sequence of cells A cell is a multi line text input field and its contents can be executed by using Shift Enter or by clicking either the Play button the toolbar or Cell Run in the menu bar The execution behavior of a cell is determined the cell s type There are four types of cells code cells markdown cells raw cells and heading cells Every cell starts off being a code cell but its type can be changed by using a dropdown on the toolbar which will be Code initially or via keyboard shortcuts Code cells A code cell allows you to edit and write new code with full syntax highlighting and tab completion By default the lan
88. configured via a file which defines the behavior of the library the details of the syntax for this can be found in the readline documentation available with your system or on the Internet Python doesn t read this file if it exists directly but it does support passing to readline valid options via a simple interface In brief you can customize readline by setting the following options in your configuration file note that these options can not be specified at the command line e readline_parse_and_bind this holds a list of strings to be executed via a readline parse_and_bind command The syntax for valid commands of this kind can be found by reading the documenta tion for the GNU readline library as these commands are of the kind which readline accepts in its configuration file e readline_remove_delims a string of characters to be removed from the default word delimiters list used by readline so that completions may be performed on strings which contain them Do not change the default value unless you know what you re doing You will find the default values in your configuration file Session logging and restoring You can log all input from a session either by starting Python with the command line switch logfile foo py see here or by activating the logging at any moment with the magic function logstart Log files can later be reloaded by running them as scripts and Python will attempt to replay the log by executing
89. content status ok Ok or terror other error info here as in other messages 406 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack buffers either 1 or 2 in length a serialization of the return value of f args kwargs only populated if status is ok All engine execution and data movement is performed via apply messages 7 3 4 Control Messages Messages that interact with the engines but are not meant to execute code are submitted via the Control queue These messages have high priority and are thus received and handled before any execution requests Clients may want to clear the namespace on the engine There are no arguments nor information involved in this request so the content is empty Message type clear_request content Message type clear_reply content status ok ok or error other error info here as in other messages Clients may want to abort tasks that have not yet run This can by done by message id or all enqueued messages can be aborted if None is specified Message type abort_request content msgq_ids s 1234s epa List of msg_ids or None Message type abort_reply content status ok ok or error other error info here as in other messages The last action a client may want to do is sh
90. debugging large programs You can run any Python file with the run function to benefit from these detailed tracebacks Furthermore both normal and verbose tracebacks can be colored if your terminal supports it which makes them much easier to parse visually See the magic xmode and colors functions for details just type Zmagic These features are basically a terminal version of Ka Ping Yee s cgitb module now part of the standard Python library Input caching system Python offers numbered prompts In Out with input and output caching also referred to as input history All input is saved and can be retrieved as variables besides the usual arrow key recall in addition to the rep magic command that brings a history entry up for editing on the next command line The following GLOBAL variables always exist so don t overwrite them 4 3 IPython reference 217 IPython Documentation Release 1 2 1 An Afternoon Hack e _i _ii _iii store previous next previous and next next previous inputs e In _ih a list of all inputs _ih n is the input from line n If you overwrite In with a variable of your own you can remake the assignment to the internal list with a simple In _ih Additionally global variables named _i lt n gt are dynamically created lt n gt being the prompt counter so _i lt n gt _ih lt n gt In lt n gt For example what you typed at prompt 14 is available as _i14 _ih 14 a
91. disabled Fix for Issue 2584 Don t kill paramiko tunnels when receiving C Add psource pfile pinfo2 commands to ipdb use 4 Pythons to build 4 Windows installers 42 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 2334 PR 2561 PR 2549 PR 2525 PR 2554 PR 2404 PR 2294 PR 2233 PR 1747 PR 2530 PR 2542 PR 2534 PR 2517 PR 2511 PR 2491 PR 2462 PR 2464 PR 2490 PR 2498 PR 2527 PR 2513 PR 2529 PR 2478 PR 2519 PR 2467 PR 2500 PR 2349 PR 2503 PR 2492 PR 2480 Fix display of plain text containing multiple carriage returns before line feed Add a simple undo for cell deletion Add event to kernel execution shell reply Avoid stopping in ipdb until we reach the main script Option to limit search result in history magic command inputhook_qt4 Use QEventLoop instead of starting up the QCoreApplication Refactored Drag and Drop Support in Qt Console switch between hsplit and vsplit paging request for feedback Adding time offsets to the video Allow starting Python as python m IPython Do not unescape backslashes in Windows shellglob Improved MathJax bug fixes trigger default remote_profile_dir when profile_dir is set color is supported in ironpython Track which extensions are loaded Locate URLs in text output and convert them to hyperlinks add ZMQInteractiveShell to PEngineApp class list
92. display it since IPython automatically creates variables called _N for prompt N execution_count int data and metadata are identical to a display_data message the object being displayed is that passed to the display hook i e the x xresult of the execution data diet metadata dict Python errors When an error occurs during code execution Message type pyerr content Similar content to the execute_reply messages for the error case except the status field is omitted Kernel status This message type is used by frontends to monitor the status of the kernel Message type status content When the kernel starts to execute code it will enter the busy state and when it finishes it will enter th idle state The kernel will publish state starting exactly once at process startup execution_state busy idle starting 7 2 7 Messages on the stdin ROUTER DEALER sockets This is a socket where the request reply pattern goes in the opposite direction from the kernel to a single frontend and its purpose is to allow raw_input and similar operations that read from sys stdin on the kernel to be fulfilled by the client The request should be made to the frontend that made the execution request that prompted raw_input to be called For now we will keep these messages as simple as possible since they only mean to convey the raw_input prompt ca
93. file magics add capture for capturing stdout err Use dvipng to format sympy Matrix Fix 1px margin bouncing of selected menu item Reconnect when the websocket connection closes unexpectedly Fix a bug in renaming notebook Fix error in test suite with ip system Q add locate entry points Fix vertical offset due to bold italics and bad browser fonts re write columnize with intermediate step new completer for qtconsole Remove suspicious quotes in interactiveshell py Rmagic exceptions notebook don t care about leading prct in completion Make svg jpeg and png images resizable in notebook HTML Notebook carriage return handling take 2 Remove importlib dependency which not available in Python 2 6 Correct stack depth for variable expansion in system commands notebook deduplicate completion results Remove args kwargs handling in TryNext fix paste error messages Keep line endings in ipynb Make invalid in filenames in the Notebook JS code doc cleanup the parallel psums example a little External cleanup fix Magic menu in qtconsole split in groups Minor bind_kernel improvements Prevent jumping of window to input when output is clicked Fix 1px jumping of cells and menus in Notebook fix chained resubmissions Rmagic extension add InlineBackend to ConsoleApp class list preserve header for resubmitted tasks 94 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack
94. fill class or function scope vars paste does not retrieve clipboard contents under screen tmux on OS X Add on or Monkey patch infrastructure for HTML notebook triple quote and s at beginning of line with paste Regression in embed SSH passwordless check with OpenSSH checks for the wrong thing Change NewNotebook handler to use 30x redirect config option for disabling history store can t use parallel Reference in view map Sympy piecewise messed up rendering paste throws an exception with empty clipboard git mpr is currently broken Variable expansion in shell commands should work in substrings Run fast tests only read a list of notebooks on server and bring into browser only notebook please put python and text editor in the web only ipython Improvements to the IPython display Image object ERROR Internal Python error in the inspect module Avoid importing from IPython parallel in core Can t trigger startup code in Engines Strange behavior for backspace with lines ending with more than 4 spaces in notebook NaN in object_info_reply JSON causes parse error 214 system command requiring administrative privileges Unknown option no edit in git mpr Add documentation build to tools test_pr py Command line option for default Notebook output collapsing behavior toggle between inline and floating figures Can t start StarCluster after upgrading to Python 0 13 oct2py v gt 0 3 1 doesn t need hSpy anymore store
95. for splitting element wise operations that operate on a sequence or array For map behavior parallel functions do have a map method call pfunc seq pfunc map seq of tasks of engines 1 per engine of engines 1 per engine of remote calls of engines 1 per engine len seq argument to remote seq i j sub sequence seq i single element A quick example to illustrate the difference in arguments for the two modes In 16 dview parallel block True brats ws def echo x dessin ns return str x In 17 echo range 5 Out 17 O 1 2 3 4 In 18 echo map range 5 Out LES sc PhO TH gp TZT VS SAY See also 280 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack See the parallel and remote decorators for options 5 3 4 Calling Python functions The most basic type of operation that can be performed on the engines is to execute Python code or call Python functions Executing Python code can be done in blocking or non blocking mode non blocking is default using the View execute method and calling functions can be done via the View apply method apply The main method for doing remote execution in fact all methods that communicate with the engines are built on top of it is View apply We strive to provide the cleanest interface we can s
96. has been moved to IPython core ipapi IPython Shell and b have been split and removed as part of the refactor e Extensions has been moved to extensions and all existing extensions have been moved to either IPython quarantine or IPython deathrow IPython quarantine contains modules that we plan on keeping but that need to be updated IPython deathrow contains mod ules that are either dead or that should be maintained as third party libraries More details about this can be found here e Previous IPython been removed to 1 GUIs in IPython frontend and IPython gui are likely broken and have Python deathrow because of the refactoring in the core With proper updates these should still work Known Regressions We do our best to improve IPython but there are some known regressions in 0 11 relative to 0 10 2 First of all there are features that have yet to be ported to the new APIs and in order to ensure that all of the installed code runs for our users we have moved them to two separate directories in the source distribution quarantine and deathrow Finally we have some other miscellaneous regressions that we hope to fix as soon as possible We now describe all of these in more detail Quarantine These are tools and extensions that we consider relatively easy to update to the new classes and APIs but that we simply haven t had time for Any user who is interested in one of these is encouraged to help us by po
97. host where the engines will run 3 Use ipengine on the engine s hosts to start the engines 272 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack The only thing you have to be careful of is to tell ipengine where the ipcontroller engine json file is located There are two ways you can do this e Put ipcontroller engine jsoninthe IPYTHONDIR profile_ lt name gt security di rectory on the engine s host where it will be found automatically e Call ipengine with the file full_path_to_the_file flag The file flag works like this ipengin file path to my ipcontroller engine json Note If the controllers and engine s hosts all have a shared file system IPYTHONDIR profile_ lt name gt security is the same on all of them then things will just work SSH Tunnels If your engines are not on the same LAN as the controller or you are on a highly restricted network where your nodes cannot see each others ports then you can use SSH tunnels to connect engines to the controller Note This does not work in all cases Manual tunnels may be an option but are highly inconvenient Support for manual tunnels will be improved You can instruct all engines to use ssh by specifying the ssh server in ipcontroller engine json uel te epe 192 168 123s 5695 1 exec_key 26 4c040 587d 4a4e b58b 030b96399584 ssh
98. if the result is ready In 10 ar ready Out 10 False 282 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack ask for the result In 45 ar get 1 but wait a maximum of 1 second Timeout Error Traceback home you lt ipython input 45 7cd858bbb8e0 gt in lt module gt ae ar get 1 path to site packages IPython parallel asyncresult pyc in get self 62 63 gt 64 65 66 Timeout else raise error Timeout def ready se Error Result not raise self _exception Lf ready Error Result not ready most recent call last timeout Note Note the import inside the function This is a common model to ensure that the appropriate modules are imported where the task is run You can also manually import modules into the engine s namespace s via view execute import numpy Often it is desirable to wait until a set of AsyncResult objects are done For this there is a the method wait This method takes a tuple of AsyncResult objects or msg_ids or indices to the client s His tory and blocks until all of the associated results are ready In 72 In 73 Wait In 74 Then In 75 Out 75 dview block Fal A trivial list of Asyn pr_list dvie until all of them dview wait pr_ se cResults objects w apply_asy
99. implementation copies them to the top level msgid uuid msg_ type str parent_header dict eontent lt dice metadata dict All messages sent to or received by any Python process should have this extended structure 7 2 Messaging in IPython 387 IPython Documentation Release 1 2 1 An Afternoon Hack 7 2 5 Messages on the shell ROUTER DEALER sockets Execute This message type is used by frontends to ask the kernel to execute code on behalf of the user in a namespace reserved to the user s variables and thus separate from the kernel s own internal code and variables Message type execute_request content Source code to be executed by the kernel one or more lines eode lt GEE A boolean flag which if True signals the kernel to execut this code as quietly as possible This means that the kernel will compile the code with exec instead of single so sys displayhook will not fire forces store_history to be False and will x not broadcast exceptions on the PUB socket do any logging Sb SR SR SR SR SR k The default is False silent bool A boolean flag which if True signals the kernel to populate history The default is True if silent is False If silent is True store_history is forced to be False store_history bool A list of variable names from the user s namespace to be retrieved What returns is a
100. in Markdown cells 2 3 Issues closed in the 1 0 development cycle 57 IPython Documentation Release 1 2 1 An Afternoon Hack 3391 3360 3375 3365 2654 3389 3388 3385 3376 3374 3380 3379 3241 3334 3326 3349 3347 3343 3337 3206 3329 3327 3303 1381 3314 3306 3280 3299 3196 3293 3292 3288 Revert 3358 once fix integrated into CM Error 500 while saving IPython notebook Frequent Safari Webkit crashes zmq frontend User_expression issues Store history as plain text Ipython parallel open TCP connection created for each result returned from engine setup py failure on Python 3 Setting __module__ to None breaks pretty printing ipython qtconsole does not display the prompt on OSX simple call to kernel TaskRecord key started not set notebook conection time out magic interpreter interpretes non magic commands python3 3 Type error when launching SGE cluster in Python notebook pip3 doesn t run 2to3 Longlist support in ipdb Make pip install easy_install faster git submodules broke nightly PPA builds Copy Paste Regression in QtConsole Buggy linewrap in Mac OSX Terminal Mountain Lion Qt version check broken parallel tasks never finish under heavy load Y for equation continuations require an extra in markdown cells Error launching iPython Test failure when running on a Vagrant VM IPython utils process getoutput returns
101. in kernel s GUI support doc update Fix sympy profile to work with sympy 0 7 issue 440 Remove unused configobj and validate libraries from externals fix various tests on Windows support pylab flag with deprecation warning Docs update setup py install depends on setuptools on Windows Get help mid command Str and Bytes traitlets Handle unicode properly in IPython zmq iostream ZMQ displayhook Handle asynchronous output in Qt console Do not import deprecated functions from external decorators library New BaseIPythonApplication Zmq unicode Fix Parallel test fallback on lsof if otool not found in libedit detection Merge Python parallel streamsession into Python zmq session 2 9 Issues closed in the 0 11 development cycle 167 IPython Documentation Release 1 2 1 An Afternoon Hack 521 492 485 486 511 509 501 495 498 494 490 491 479 483 419 41T 459 475 471 474 472 460 469 464 455 448 453 452 398 381 413 370 use dict get key instead of dict key for safety from KeyErrors add QtConsoleApp using newapplication terminal Python with newapp Use newapp in parallel code Add a new line before displaying multiline strings in the Qt console i508 ignore EINTR in channel loops Better selection of Qt bindings when QT_API is not specified Check for pyd as extension for binary files QtConsole zoom adjustments fix UnicodeEncodeError writing SVG string t
102. in notebook names PR 3918 added check_for_tornado closes 3916 PR 3917 change docs examples refs to be just examples PR 3908 what s new tweaks 24 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 3896 PR 3911 PR 3901 PR 3891 PR 3892 PR 3888 PR 3882 PR 3877 PR 3878 PR 3855 PR 3879 PR 3870 PR 3876 PR 3872 PR 3871 PR 3873 PR 3868 PR 3865 PR 3861 PR 3864 PR 3834 PR 3840 PR 3859 PR 3854 PR 3857 PR 3845 PR 3846 PR 3850 PR 3844 PR 3830 PR 3848 PR 3836 two column quickhelp dialog closes 3895 explicitly load python mode before Python mode don t force relative path fix 3897 fix 3889 Fix documentation of Kernel stop_channels posixify paths for Windows latex quick fix for 3881 don t use shell True in PDF export minor template loading cleanup nbconvert Filter tests finish 3870 Fix for converting notebooks that contain unicode characters Update parallel_winhpc rst removing vim ipython since it has it s own repo updating docs remove old examples update CodeMirror component to 3 15 Escape filename for pdflatex in nbconvert remove old external js add keyboard shortcut to docs This PR fixes a few issues with nbconvert tests prevent profile_dir from being undefined Add An Afternoon Hack to docs Catch errors filling readline history on startup Delete extr
103. instance whose name begins with an uppercase character is assumed to be a sub configuration and a new empty Config instance is dynamically created for that attribute This allows deeply hierarchical information created easily c Foo Bar value on the fly 6 1 3 Configuration files inheritance Let s say you want to have different configuration files for various purposes Our configuration sys tem makes it easy for one configuration file to inherit the information in another configuration file The 6 1 Overview of the IPython configuration system 349 IPython Documentation Release 1 2 1 An Afternoon Hack load_subconfig command can be used in a configuration file for this purpose Here is a simple example that loads all of the values from the file base_config py base_config py c get_config c MyClass name coolname c MyClass ranking 100 into the configuration file main_config py main_config py c get_config Load everything from base_config py load_subconfig base_config py Now override one of the values c MyClass name bettername In a situation like this the load_subconfig makes sure that the search path for sub configuration files is inherited from that of the parent Thus you can typically put the two in the same directory and everything will just work You can also load configuration files by profile for instance load_subconfig
104. interpreter e __ two underscores next previous e ___ three underscores next next previous Additionally global variables named _ lt n gt are dynamically created lt n gt being the prompt counter such that the result of output lt n gt is always available as _ lt n gt don t use the angle brackets just the number e g _21 These variables are also stored in a global dictionary not a list since it only has entries for lines which returned a result available under the names _oh and Out similar to _ih and In So the output from line 12 can be obtained as _12 Out 12 or _oh 12 If you accidentally overwrite the Out variable you can recover it by typing Out _oh at the prompt This system obviously can potentially put heavy memory demands on your system since it prevents Python s garbage collector from removing any previously computed results You can control how many results are kept in memory with the option at the command line or in your configuration file cache_size If you set it to 0 the whole system is completely disabled and the prompts revert to the classic gt gt gt of normal Python 218 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack Directory history Your history of visited directories is kept in the global list _dh and the magic cd command can be used to go to any entry in that list The dhist command allows you to view thi
105. into console functions using tab qtconsoleapp parse_command_line doen t like existing anymore Qt console pylab inline and getfigs etc unwanted unicode passed to args Users should see tooltips for all buttons in the notebook UI ipython crashes if atexit handler raises exception use of Tracer when debugging works but gives error messages debugger does not print error message by default in 0 11 2 7 Issues closed in the 0 12 development cycle 149 IPython Documentation Release 1 2 1 An Afternoon Hack aA 749 890 885 881 744 865 J29 817 799 732 662 650 829 874 640 624 871 history of multiline entries IPython parallel test failure under Windows 7 and XP ipclusterapp py helep ws hostname alias not recognized by notebook Missing manual pdf cannot create notebook in offline mode if mathjax not installed Make tracebacks from paste show the code exception unicode handling in run is faulty in qtconsole iPython crashed edit magic not working on windows xp in qtconsole QTConsole wrongly promotes the index of the input line on which user presses Enter ipython test failures on Mac OS X Lion Handle bad config files better We should not insert new lines after all print statements in the notebook ipython qtconsole pyzmq Version Comparison matplotlib macosx windows don t respond in qtconsole ipython intermittently segfaults when figure is closed Mac OS X Notebook cra
106. is a key necessary to submit requests but due to the lack of encryption it does not provide significant security if loopback traffic is compromised In our architecture the controller is the only process that listens on network ports and is thus the main point of vulnerability The standard model for secure connections is to designate that the controller listen on localhost and use ssh tunnels to connect clients and or engines 262 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack To connect and authenticate to the controller an engine or client needs some information that the controller has stored in a JSON file Thus the JSON files need to be copied to a location where the clients and engines can find them Typically this is the ipython profile_default security directory on the host where the client engine is running which could be a different host than the controller Once the JSON files are copied over everything should work fine Currently there are two JSON files that the controller creates ipcontroller engine json This JSON file has the information necessary for an engine to connect to a con troller ipcontroller client json The client s connection information This may not differ from the engine s but since the controller may listen on different ports for clients and engines it is stored separately ipcontroller client json will look something like this
107. it very easy to start any script under the control of pdb regardless of whether you have wrapped it into a main function or not For this simply type run d myscript at an Python prompt See the run command s documentation via run or in Sec magic for more details including how to control where pdb will stop execution first For more information on the use of the pdb debugger read the included pdb doc file part of the standard Python distribution On a stock Linux system it is located at usr lib python2 3 pdb doc but the easiest way to read it is by using the help function of the pdb module as follows in an Python prompt In 1 import pdb In 2 pdb help This will load the pdb doc document in a file viewer for you automatically Automatic invocation of pdb on exceptions IPython if started with the pdb option or if the option is set in your config file can call the Python pdb debugger every time your code triggers an uncaught exception This feature can also be toggled at any time with the pdb magic command This can be extremely useful in order to find the origin of subtle bugs because pdb opens up at the point in your code which triggered the exception and while your program is at this point dead all the data is still available and you can walk up and down the stack frame and understand the origin of the problem Furthermore you can use these debugging facilities both with
108. kernel model and are forwarded through a ZMQ Queue device The controller receives all messages and replies in these channels and saves results for future use 7 3 1 The Controller The controller is the central collection of processes in the Python parallel computing model It has two major components The Hub A collection of Schedulers 7 3 2 The Hub The Hub is the central process for monitoring the state of the engines and all task requests and results It has no role in execution and does no relay of messages so large blocking requests or database actions in the Hub do not have the ability to impede job submission and results Registration ROUTER The first function of the Hub is to facilitate and monitor connections of clients and engines Both client and engine registration are handled by the same socket so only one ip port pair is needed to connect any number of connections and clients Engines register with the zmq IDENTITY of their two DEALER sockets one for the queue which receives execute requests and one for the heartbeat which is used to monitor the survival of the Engine process Message type registration_request content uuid abcd 1234 the zmq IDENTITY of the engine s sockets Note these are always the same at least for now The Controller replies to an Engine s registration request with the engine s integer ID and all the remaining connection information for co
109. kernel must figure out the completion based on line and cursor_pos text str The full line such as print a is This allows completers to make decisions that may require information about more than just the current word Line str The entire block of text where the line is This may be useful in the case of multiline completions where more context may be needed Note iff in practice this field proves unnecessary remove it to lighten the messages block str or null None The position of the cursor where the user hit TAB on the line cursor pos int Message type complete_reply content The list of all matches to the completion request such as a isalnum a isalpha for the abov xampl matches list the substring of the matched text this is typically the common prefix of the matches and the text that is already in the block that would be replaced by the full completion This would be a is in the above exampl 1 matched_text str status should be ok unless an exception was raised during the request in which case it should be error along with the usual error message content in other messages status Tok History For clients to explicitly request history from a kernel The kernel has all the actual execution history stored in a single location so clients can request it from the kernel when
110. least on mac 1192 Invalid JSON data 1196 docs looks like a file path might have been accidentally pasted in the middle of a word 1189 Right justify of in prompt in variable prompt size configurations 1185 ipython console not work proper with stdout 1191 profile startup files not executed with notebook 1190 Fix link to Chris Fonnesbeck blog post about 0 11 highlights 1174 Remove install_default_config and install_profiles 2 6 0 12 Series 2 6 1 Release 0 12 1 Python 0 12 1 is a bugfix release of 0 12 pulling only bugfixes and minor cleanup from 0 13 timed for the Ubuntu 12 04 LTS release See the list of fixed issues for specific backported issues 2 6 2 Release 0 12 Python 0 12 contains several major new features as well as a large amount of bug and regression fixes The 0 11 release brought with it a lot of new functionality and major refactorings of the codebase by and large this has proven to be a success as the number of contributions to the project has increased dramatically proving that the code is now much more approachable But in the refactoring inevitably some bugs were introduced and we have also squashed many of those as well as recovered some functionality that had been temporarily disabled due to the API changes The following major new features appear in this version An interactive browser based Notebook with rich media support A powerful new interface puts Python in
111. load_balanced_view scatter id so id 0 1 2 on engines 0 1 2 dv scatter id rc ids flatten True print Engine IDs dv id create a Reference to id This will be a different value on each engin ref parallel Reference id print sleeping for id seconds on each engine tic time time ar dv apply time sleep ref for i r in enumerate ar print i 3 3f i time time tic def sleep_here t import time time sleep t return id t one call per task print running with one call per task amr v map sleep_here 01lxt for t in range 100 tic time time for i r in enumerate amr print task i on engine i 3f i r 0 time time tic print running with four calls per task 304 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack with chunksize we can have four calls per task amr v map sleep_here 01 t for t in range 100 chunksize 4 tic time time for i r in enumerate amr print task i on engine i 3f i r 0 time time tic print running with two calls per task with unordered results We can even iterate through faster results first with ordered False amr v map sleep_here 01 t for t in range 100 0 1 ordered False chunksize 2 tic time time for i r in enumerate amr print Slept 2fs on engine i 3f r 1 r 0
112. make a plot it will show up in your document as if you had called display fig The inline backend can use either SVG or PNG figures PNG being the default It also supports the special key retina which is 2x PNG for high DPI displays To switch between them set the InlineBackend figure_format configurable in a config file or via the config magic In 10 config InlineBackend figure_format svg Note Changing the inline figure format also affects calls to display above even if you are not using the inline backend for all figures By default Python closes all figures at the completion of each execution This means you don t have to manually close figures which is less convenient when figures aren t attached to windows with an obvious close button It also means that the first matplotlib call in each cell will always create a new figure In 11 plt plot range 100 lt single line plot gt In 12 plt plot 1 3 2 lt another single line plot gt 236 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack However it does prevent the list of active figures surviving from one input cell to the next so if you want to continue working with a figure you must hold on to a reference to it In 11 Eig get fig plot rand 100 lt plot gt In 12 fig title Random Title lt redraw plo
113. methods 1727 terminate kernel after embed_kernel tests 1737 add HistoryManager to ipapp class list 945 Open a notebook from the command line 1686 ENH Open a notebook from the command line 1709 fixes 1708 failing test in arg_split on windows 1718 Use CRegExp trait for regular expressions 1729 Catch failure in repr for whos 1726 use eval for command line args instead of exec 1723 scatter gather fail with targets all 1724 fix scatter gather with targets all 1725 add no ff to git pull in test_pr 1722 unicode exception when evaluating expression with non ascii characters 1721 Tooltip completer js refactor 1657 Add wait optional argument to hooks editor 123 Define sys ps 1 2 1717 Define generic sys ps 1 2 3 for use by scripts 1442 cache size issue in qtconsole 1691 Finish PR 1446 1446 Fixing Issue 1442 1710 update MathJax CDN url for https 73 81 Autocall fails if first function argument begins with or 1713 Make autocall regexp s configurable 211 paste command not working 1703 Allow TryNext to have an error message without it affecting the command chain 1714 minor adjustments to test_pr 2 5 Issues closed in the 0 13 development cycle 113 IPython Documentation Release 1 2 1 An Afternoon Hack 1509 New tooltip for notebook 1697 Major refactoring of the Notebook Kernel and CodeCell JavaScript
114. msg the content dict is keyed by msg_ids values are the result messages there will be none of these if statusonly True pending msg_id msg_ids still pending completed msg_id list of completed msg ids buffers bufs the buffers that contained the results of the objects this will be empty if no messages are complete or iff statusonly is True For memory management purposes Clients can also instruct the hub to forget the results of messages This can be done by message ID or engine ID Individual messages are dropped by msg_id and all messages completed on an engine are dropped by engine ID This may no longer be necessary with the mongodb based message logging backend If the msg_ids element is the string a11 instead of a list then all completed results are forgotten Message type purge_request content msg ids adi ade cssdy FF List of msg 1ds or ali engine_ids 0 2 4 list of engine IDs The reply to a purge request is simply the status ok if the request succeeded or an explanation of why it failed such as requesting the purge of a nonexistent or pending message Message type purge_reply content status ok or error 7 3 3 Schedulers There are three basic schedulers e Task Scheduler e MUX Scheduler e Control Scheduler The MUX and Control schedulers are simple MonitoredQueue
115. needed Message type history_request 394 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack content If True also return output history in the resulting dict output lt bool If True return the raw input history else the transformed input raw bool So far this can be range tail or search hist_access_type str If hist_access_type is range get a range of input cells session can be a positive session number or a negative number to count back from the current session session int start and stop are line numbers within that session esttare Int stop ant If hist_access_type is tail or search get the last n cells nm lt 2 ane If hist_access_type is search get cells matching the specified glob pattern with and as wildcards pattern str If hist_access_type is search and unique is true do not include duplicated history Default is false unique bool New in version 4 0 The key unique for history_request Message type history_reply content A list of 3 tuples either session line_number input or session line_number input output depending on whether output was False or True respectively history 2 List Connect When a client connects to the request reply socket of the kernel it can issue a conn
116. new features with examples and screenshots We encourage you to read it as it provides an illustrated high level overview complementing the detailed feature breakdown in this document A quick summary of the major changes see below for details Standalone Qt console a new rich console has been added to IPython started with ipython qtconsole In this application we have tried to retain the feel of a terminal for fast and efficient workflows while adding many features that a line oriented terminal simply can not support such as inline figures full multiline editing with syntax highlighting graphical tooltips for function calls and much more This development was sponsored by Enthought Inc See below for details High level parallel computing with ZeroMQ Using the same architecture that our Qt console is based on we have completely rewritten our high level parallel computing machinery that in prior versions used the Twisted networking framework While this change will require users to update their codes the improvements in performance memory control and internal consistency across our codebase convinced us it was a price worth paying We have tried to explain how to best proceed with this update and will be happy to answer questions that may arise A full tutorial describing these features was presented at SciPy 11 more details below New model for GUI plotting support in the terminal Now instead of the various Xthread flags we had
117. new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 1477 Issues 27 PR 1364 PR 1369 PR 1370 PR 1374 PR 1377 PR 1396 PR 1402 PR 1413 PR 1414 PR 1430 PR 1445 PR 1463 PR 1495 PR 1496 PR 1514 PR 1517 PR 1538 PR 1599 PR 1602 PR 1607 avoid jsonlib returning Decimal load header with engine id when engine dies in TaskScheduler allow draft76 websockets Safari remove calls to meaningless ZMQStream on_err Saving non ascii history Fix for tb magic fix symlinked home issue for FreeBSD get_home_dir expands symlinks adjust test accordingly ignore errors in shell var_expand Fix for tornado check for tornado lt 1 1 0 Don t build sphinx docs for sdists Fix completion when importing modules in the cwd fix dangling buffer in Python parallel util BUG Fix pretty printing for overzealous objects BUG LBYL when clearing the output history on shutdown DOC Fix references to Python lib pretty instead of the old location Fix indentation bug in IPython lib pretty py store git commit hash in utils _sysinfo instead of hidden data file Fix for run d in Python 3 Fix env for Python 3 cleanup sqlitedb temporary db file after tests 676 IPython embed from ipython crashes twice on exit 846 Autoreload extension doesn t work with Python 3 2 1187 misc notebook connection file cleanup first heartbeat startup flush 1191 profile star
118. notebooks and standard py Python source code files into the notebook list area When starting a notebook server from the command line you can also open a particular notebook directly bypassing the dashboard with ipython notebook my_notebook ipynb The ipynb extension is assumed if no extension is given When you are inside an open notebook the File Open menu option will open the dashboard in a new browser tab to allow you to open another notebook from the notebook directory or to create a new notebook Note You can start more than one notebook server at the same time if you want to work on notebooks in different directories By default the first notebook server starts on port 8888 and later notebook servers search for ports near that one You can also manually specify the port with the port option Creating a new notebook document A new notebook may be created at any time either from the dashboard or using the File New menu option from within an active notebook The new notebook is created within the same directory and will open in a new browser tab It will also be reflected as a new entry in the notebook list on the dashboard Opening notebooks An open notebook has exactly one interactive session connected to an Python kernel which will execute code sent by the user and communicate back results This kernel remains active if the web browser window is closed and reopening the same notebook from the dashboard wil
119. now be displayed with a custom traceback by defining a _render_traceback_ method which returns a list of strings each containing one line of the traceback A new command ipython history trim can be used to delete everything but the last 1000 entries in the history database ___file__ is defined in both config files at load time and ipy files executed with run S logstart and logappend are no longer broken Add glob expansion for run e g run g script py txt Expand variables foo in Cell Magic argument line By default iptest will exclude various slow tests All tests can be run with iptest all SQLite history can be disabled in the various cases that it does not behave well edit works on interactively defined variables editor hooks have been restored from quarantine enabling TextMate as editor etc The env variable PYTHONSTARTUP is respected by IPython The Smatplot1lib magic was added which is like the old Spylab magic but it does not import anything to the interactive namespace It is recommended that users switch to smatplotlib and explicit imports The matplotlib command line flag was also added It invokes the new Smatplotlib magic and can be used in the same way as the old pylab flag You can either use it by itself as a flag matplotlib or you can also pass a backend explicitly matplotlib qt or matplot lib wx etc Backwards incompatible changes Calling InteractiveShell prefilt
120. of the script is for starting things on localhost Eventually when ipcluster is able to start things on other hosts we will put security back cd foo searches directory history for string foo and jumps to that dir Last part of dir name is checked first If no matches for that are found look at the whole path Bug fixes The Windows installer has been fixed Now all IPython scripts have bat versions created Also the Start Menu shortcuts have been updated The colors escapes in the multiengine client are now turned off on win32 as they don t print correctly The IPython kernel scripts ipengine script was exec ing mpi_import_statement incor rectly which was leading the engine to crash when mpi was enabled A few subpackages had missing ___ init__ py files The documentation is only created if Sphinx is found Previously the setup py script would fail if it was missing Greedy cd completion has been disabled again it was enabled in 0 8 4 as it caused problems on certain platforms Backwards incompatible changes The clusterfile options of the ipcluster command has been removed as it was not working and it will be replaced soon by something much more robust The IPython kernel configuration now properly find the user s Python directory In ipapi the make_user_ns function has been replaced with make_user_namespaces to support dict subclasses in namespace creation e TPython kernel client Tas
121. on engines where names you specify are importable 5 5 The IPython task interface 297 IPython Documentation Release 1 2 1 An Afternoon Hack In 10 require numpy zmq def myfunc P return dostuff Now any time you apply myfunc the task will only run on a machine that has numpy and pyzmq available and when myfunc is called numpy and zmq will be imported depend The depend decorator lets you decorate any function with any other function to evaluate the dependency The dependency function will be called at the start of the task and if it returns False then the dependency will be considered unmet and the task will be assigned to another engine If the dependency returns anything other than False the rest of the task will continue In 10 def platform_specific plat E import sys seiat return sys platform plat In 11 depend platform specific darwin def mactask spe ees do_mac_stuff In 12 depend platform_specific nt def wintask LEDES do_windows_stuff In this case any time you apply mytask it will only run on an OSX machine depend is just like apply in that it has a depend f args xkwargs signature dependents You don t have to use the decorators on your tasks if for instance you may want to run tasks with a single function but varying dependencies you can directly construct the dependent obje
122. original 2001 codebase We have also introduced a new type of magic function prefixed with instead of which operates at the whole cell level A cell magic receives two arguments the line it is called on like a line magic and the body of the cell below it Cell magics are most natural in the notebook but they also work in the terminal and qt console with the usual approach of using a blank line to signal cell termination For example to time the execution of several statements o Sstimeit x 0 setup for i in range 100000 x 1x 2 This is particularly useful to integrate code in another language and cell magics already exist for shell scripts Cython R and Octave Using script usr bin foo you can run a cell in any interpreter that accepts code via stdin Another handy cell magic makes it easy to write short text files b file save to here txt The following cell magics are now included by default all those that use special interpreters Perl Ruby bash etc assume you have the requisite interpreter installed e run cell body with the underlying OS shell this is similar to prefixing every line in the cell with e bash run cell body under bash Scapture capture the output of the code in the cell and stderr as well Useful to run codes that produce too much output that you don t even want scrolled H e ol file save cell body as a file ol perl1 run cell bo
123. places and installs the command line script to the appropriate places Then if you want to update your Python at any time just do S git pull 3 3 Installing IPython itself 197 IPython Documentation Release 1 2 1 An Afternoon Hack 3 4 Basic optional dependencies There are a number of basic optional dependencies that most users will want to get These are e readline for command line editing tab completion etc e nose to run the Python test suite e pexpect to use things like irunner If you are comfortable installing these things yourself have at it otherwise read on for more details 3 4 1 readline As indicated above on Windows PyReadline is a mandatory dependency PyReadline is a separate Win dows only implementation of readline that uses native Windows calls through ct ypes The easiest way of installing PyReadline is you use the binary installer available here On OSX if you are using the built in Python shipped by Apple you will be missing a full readline implemen tation as Apple ships instead a library called 1ibedit that provides only some of readline s functionality While you may find libedit sufficient we have occasional reports of bugs with it and several developers who use OS X as their main environment consider libedit unacceptable for productive regular use with Python Therefore we strongly recommend that on OS X you get the full readline module We will not consider
124. plotting is performed you must execute the smatplotlib magic command This performs the necessary behind the scenes setup for Python to work correctly hand in hand with matplotlib it does not however actually execute any Python import commands that is no names are added to the namespace If the Smatplotlib magic is called without an argument the output of a plotting command is displayed using the default mat plot 1ib backend in a separate window Alternatively the backend can be explicitly requested using for example smatplotlib gtk A particularly interesting backend provided by IPython is the inline backend This is available only for the Python Notebook and the Python QtConsole It can be invoked as follows 250 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack smatplotlib inline With this backend the output of plotting commands is displayed inline within the notebook directly below the code cell that produced it The resulting plots will then also be stored in the notebook document See also Plotting with Matplotlib example notebook 4 6 7 Configuring the IPython Notebook The notebook server can be run with a variety of command line arguments To see a list of available options enter ipython notebook help Defaults for these options can also be set by creating a file named ipython_notebook_config pyin your Python p
125. protocol 2589 Creation of 300 MPI spawned engines causes instability in ipcluster 2672 notebook inline option without pylab 2673 Indefinite Articles amp Traitlets 2705 Notebook crashes Safari with select and drag 2721 dreload kills ipython when it hits zmq 64 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 2806 2794 2793 1085 2760 2780 2772 2754 2764 2765 2540 2394 2440 1814 2759 2615 2748 1661 2742 2739 2588 2550 ables 2675 2419 1523 2719 2715 2555 2707 2696 2684 ipython parallel doesn t discover globals under Python 3 3 _exit_code behaves differently in terminal vs ZMQ frontends IPython parallel issue with pushing pandas TimeSeries In process kernel for Qt frontend IndexError list index out of range with Python 3 2 Save and load notebooks from github AttributeError Client object has no attribute kill Fail to send class definitions from interactive session to engines namespaces TypeError while using cd name _ file is not defined Wrap tooltip if line exceeds threshold Startup error on ipython qtconsole version 0 13 and 0 14 dev IPEP 4 Python 3 Compatibility __file__ is not defined when file end with ipy R magic extension interferes with tab completion Small change needed to rmagic extension collapse parts of a html notebook Yopaste
126. raises ERROR Session line number was not unique in database 7 Update the changes doc to reflect the kernel config work 303 Users should be able to scroll a notebook w o moving the menu buttons 322 Embedding an interactive Python shell 321 debug broken in master 287 Crash when using macros in sqlite history branch 55 Can t edit files whose names begin with numbers 284 In variable no longer works in 0 11 92 Using multiprocessing module crashes parallel iPython 262 Fail to recover history after force kill 320 Tab completing re search objects crashes Python 317 IPython kernel parallel map issues 197 ipython qtconsole unicode problem in magic Is 305 more readline shortcuts in qtconsole 314 Multi line multi block cells can t be executed 308 Test suite should set sqlite history to work in memory 202 Matplotlib native MacOSX backend broken in pylab mode 196 Python can t deal with unicode file name 25 unicode bug encoding input 290 try except else clauses can t be typed code input stops too early 178 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 43 Implement SSH support in ipcluster 6 Update the Sphinx docs for the new ipcluster 9 Getting DeadReferenceError Calling Stale Broker after ipcontroller restart 132 Ipython prevent south from working 27 generics complete_object broken 60 Improve absolut
127. rich representation of each variable dict keyed by name See the display_data content for the structure of the representation data user_variables list Similarly a dict mapping names to expressions to be evaluated in the user s dict user_expressions dict Some frontends e g the Notebook do not support stdin requests If raw_input is called from code executed from such a frontend a StdinNotImplementedError will be raised allow_stdin True The code field contains a single string possibly multiline The kernel is responsible for splitting this into one or more independent execution blocks and deciding whether to compile these in single or exec mode see below for detailed execution semantics The user_ fields deserve a detailed explanation In the past Python had the notion of a prompt string that allowed arbitrary code to be evaluated and this was put to good use by many in creating prompts that displayed system status path information and even more esoteric uses like remote instrument status acquired over the network But now that Python has a clean separation between the kernel and the clients 388 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack the kernel has no prompt knowledge prompts are a frontend side feature and it should be even possible for different frontends to display different prompts while interacting with t
128. simultaneously This opens the door for lots of interesting things The Hub The center of an Python cluster is the Hub This is the process that keeps track of engine connections schedulers clients as well as all task requests and results The primary role of the Hub is to facilitate queries of the cluster state and minimize the necessary information required to establish the many connections involved in connecting new clients and engines Schedulers All actions that can be performed on the engine go through a Scheduler While the engines themselves block when user code is run the schedulers hide that from the user to provide a fully asynchronous interface to a set of engines IPython client and views There is one primary object the Client for connecting to a cluster For each execution model there is a corresponding View These views allow users to interact with a set of engines through the interface Here are the two default views e The Direct View class for explicit addressing e The LoadBalancedView class for destination agnostic scheduling Security Python uses ZeroMQ for networking which has provided many advantages but one of the setbacks is its utter lack of security ZeroMQ By default no Python connections are encrypted but open ports only listen on localhost The only source of security for Python is via ssh tunnel Python supports both shell openssh and paramiko based tunnels for connections There
129. single person or institution Instead it is the collective copyright of the entire Python Development Team If individual contributors want to maintain a record of what changes contributions they have specific copyright on they should indicate their copyright in the commit message of the change when they commit the change to one of the Python repositories Any new code contributed to Python must be licensed under the BSD license or a similar MIT open source license 9 3 4 Miscellaneous Some files DPyGetOpt py for example may be licensed under different conditions Ultimately each file indicates clearly the conditions under which its author authors have decided to publish the code Versions of Python up to and including 0 6 3 were released under the GNU Lesser General Public License LGPL available at http www gnu org copyleft lesser html 9 3 License and Copyright 425 IPython Documentation Release 1 2 1 An Afternoon Hack 426 Chapter 9 About IPython Bibliography ZeroMQ ZeroMQ http www zeromq org MongoDB MongoDB database http www mongodb org PBS Portable Batch System http www openpbs org SSH SSH Agent http en wikipedia org wiki ssh agent MPI Message Passing Interface http www unix mcs anl gov mpi mpi4py MPI for Python mpi4py http mpi4py scipy org OpenMPI Open MPI http www open mpi org PyTrilinos PyTrilinos http trilinos sandia gov packages pytrilinos RFC5246
130. socket for querying the Hub for state information Heartbeat Client s n O M a a a a a a i i Schedulers Hub i i i i i i i i i i i 1 i i L E i i i L i i i il LI Engine s ZMO_ FORWARDER Heartbeat Fig 7 3 The heartbeat sockets The heartbeat process has been described elsewhere To summarize the Heartbeat Monitor publishes a dis tinct message periodically via a PUB socket Each engine has a zmq FORWARDER device with a SUB socket for input and D EA I ER socket for output The SUB socket is connected to the PU B socket labeled ping and 7 4 Connection Diagrams of The IPython ZMQ Cluster 411 IPython Documentation Release 1 2 1 An Afternoon Hack the DEALER is connected to the ROUTER labeled pong This results in the same message being relayed back to the Heartbeat Monitor with the addition of the DEALER prefix The Heartbeat Monitor receives all the replies via an ROUTER socket and identifies which hearts are still beating by the zmq IDENTITY prefix of the DEALER sockets which information the Hub uses to notify clients of any changes in the available engines Schedulers Contral Client s Task Direct Hub Schedulers Balanced Q Control Engine s Apply ugue Queue Fig 7 4 Control message scheduler on the left execution apply schedulers on the right The controller has a
131. sshx sh 276 Calling sip setapi does not work if app has already imported from PyQt4 277 matplotlib image imgshow from 10 1 segfault 288 Incorrect docstring in zmq kernelmanager py 286 Fix Python Shell compatibility layer 99 blank lines in history 129 psearch TypeError expected string or buffer 190 Add option to format float point output 180 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 246 e 176 e 279 e 119 e 271 e 270 e 166 a 173 Application not conforms XDG Base Directory Specification 48 IPython should follow the XDG Base Directory spec for configuration Make client side history persistence readline independent Backtraces when using ipdb do not respect colour LightBG setting Broken type filter in magic_who_ls Intermittent problem with print output in Qt console Small typo in IPython developer s guide Add keyboard accelerators to Qt close dialog asymmetrical ctrl A ctrl E behavior in multiline e 45 Autosave history for robustness e 162 e 161 e 263 e 256 258 248 e 247 e 245 e 241 e 213 117 105 210 e 209 e 206 e 194 e 186 make command history persist in ipythonqt make ipythonqt exit without dialog when exit is called ipython numpy Some test errors reset docstring ipython 0 10 allow caching to avoid matplotlib object referrences Can t open and read files after upgrade from 0 10
132. start Python with matplotlib support use the matplotlib switch If Python is already running you can run the matplotlib magic If no arguments are given Python will automatically detect your choice of matplotlib backend You can also request a specific backend with Smatplotlib backend where backend must be one of tk qt wx gtk osx In the web notebook and Qt console inline is also a valid backend value which produces static figures inlined inside the application window instead of matplotlib s interactive figures that live in separate windows 4 3 9 Interactive demos with IPython Python ships with a basic system for running scripts interactively in sections useful when presenting code to audiences A few tags embedded in comments so that the script remains valid Python code divide a file into separate blocks and the demo can be run one block at a time with Python printing with syntax highlighting the block before executing it and returning to the interactive prompt after each block The interactive namespace is updated after each block is run with the contents of the demo s namespace 228 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack This allows you to show a piece of code run it and then execute interactively commands based on the variables just created Once you want to continue you simply execute the next block of the de
133. still bugs about IndentationError and says to use paste Octavemagic fails to deliver inline images in Python on Windows wiki ipython org contaminated with prescription drug spam Link error while executing code from cython example notebook Rpush magic doesn t find local variables and doesn t support comma separated lists of vari Markdown html blockquote need css TerminalInteractiveShell __init__Q ignores value of ipython_dir argument Better LaTeX printing in the qtconsole with the sympy profile ipython fails with pkg_resources DistributionNotFound ipython 0 13 url crashes nbviewer ipython org import module completion on MacOSX Problem installing the new version of Python in Windows SymPy magic bug in Python Notebook pretty print broken for types created with PyType_FromSpec 2 3 Issues closed in the 1 0 development cycle 65 IPython Documentation Release 1 2 1 An Afternoon Hack 2533 2661 2679 2646 2587 2660 2665 2611 2658 2655 2651 2645 2637 2624 2608 1275 2613 rmagic breaks on Windows Qtconsole tooltip is too wide when the function has many arguments ipython3 qtconsole via Homebrew on Mac OS X 10 8 pyqt pyside import error pylab_not_importable cython magic pops 2 CLI windows upon execution on Windows Certain arguments h help version never passed to scripts run with ipython Missing docs for rmagic and some other extensions Travis
134. stop_spin_thread for running spin in a background thread to keep zmq queue clear This can be used to ensure that timing information is as accurate as possible at the cost of having a background thread active Set TaskScheduler hwm default to 1 instead of 0 1 has more predictable intuitive behavior if often slower and thus a more logical default Users whose workloads require maximum throughput and are largely homogeneous in time per task can make the optimization themselves but now the behavior will be less surprising to new users PR 1294 Kernel Engine unification This is mostly work under the hood but it is actually a major achievement for the project that has deep implications in the long term at last we have unified the main object that executes as the user s interactive shell which we refer to as the Python kernel with the objects that run in all the worker nodes of the parallel computing facilities the Python engines Ever since the first implementation of Python s parallel code back in 2006 we had wanted to have these two roles be played by the same machinery but a number of technical reasons had prevented that from being true In this release we have now merged them and this has a number of important consequences e It is now possible to connect any of our clients qtconsole or terminal console to any individual parallel engine with the exact behavior of working at a regular Python console qtconsol
135. switches for interpretation that are made at the task level These are specified via keyword arguments to the client s apply method after follow You may want to run a task after a given set of dependencies have been run and or run it where another set of dependencies are met To support this every task has an after dependency to restrict time and a follow dependency to restrict destination timeout You may also want to set a time limit for how long the scheduler should wait before a task s dependencies are met This is done via a timeout which defaults to 0 which indicates that the task should never timeout If the timeout is reached and the scheduler still hasn t been able to assign the task to an engine the task will fail with a DependencyTimeout Note Dependencies only work within the task scheduler You cannot instruct a load balanced task to run after a job submitted via the MUX interface The simplest form of Dependencies is with all True success True failure False In these cases you can skip using Dependency objects and just pass msg_ids or AsyncResult objects as the follow and after key words to client apply In 14 client block False In 15 ar lview apply f args kwargs In 16 ar2 lview apply f2 In 17 with lview temp_flags after ar ar2 ar3 lview apply f3 In 18 with lview temp_flags follow ar timeout 2 5 EE ar4 lview apply f3 See also
136. time time tic That is to say if you treat an AsyncMapResult as if it were a list of your actual results it should behave as you would expect with the only difference being that you can start iterating through the results before they have even been computed This lets you do a dumb version of map reduce with the builtin Python functions and the only difference between doing this locally and doing it remotely in parallel is using the asynchronous view map instead of the builtin map Here is a simple one line RMS root mean square implemented with Python s builtin map reduce In 38 X np linspace 0 100 In 39 from math import sqrt In 40 add lambda a b a b In 41 sq lambda x xxx In 42 sqrt reduce add map sq X len X Out 42 58 028845747399714 In 43 sqrt reduce add view map sq X len X Out 43 58 028845747399714 To break that down 1 map sq X Compute the square of each element in the list locally or in parallel 2 reduce add sqX len X compute the mean by summing over the list or AsyncMapRe sult and dividing by the size 3 take the square root of the resulting number See also When AsyncResult or the AsyncMapResult don t provide what you need for instance handling individual results as they arrive but with metadata you can always just split the original result s msg_ids attribute and handle them as you like For an example of th
137. to 0 10 0 ipython Stackless Magic save and macro missing newlines line ranges don t match prompt numbers exit hangs on terminal version of Python ipython pylab no longer plots interactively on 0 10 1 4 wx frontend don t display well commands output 5 Is command not supported in ipythonx wx frontend 1 Document winhpcjob py and launcher py 83 Usage of testing util DeferredTestCase should be replace with twisted trial unittest TestCase Redesign how Component instances are tracked and queried 47 IPython kernel client cannot be imported inside an engine Refactor the task dependencies system 0 10 1 doc mistake New Python Sphinx directive error can t activate Python parallel magics Buggy linewrap in Mac OSX Terminal sudo lt command gt displays password in plain text edit issue under OS X 10 5 Python 0 10 1 2 9 Issues closed in the 0 11 development cycle 181 IPython Documentation Release 1 2 1 An Afternoon Hack 11 Create a daily build PPA for ipython 144 logo missing from sphinx docs 181 cls command does not work on windows 169 Kernel can only be bound to localhost 36 tab completion does not escape e 177 Report tracebacks of interactively entered input e 148 dictionary having multiple keys having frozenset fails to print on iPython e 160 magic_gui throws TypeError when gui magic is used e 150 History entries ending with parentheses corrupt command line on OS X 10 6
138. to it are reflected immediately unlike imported modules which have to be specifically reloaded Python also includes dreload a recursive reload function run has special flags for timing the execution of your scripts t or for running them under the control of either Python s pdb debugger d or profiler p The edit command gives a reasonable approximation of multiline editing by invoking your favorite editor on the spot Python will execute the code you type in there as if it were typed interactively 204 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack Debugging After an exception occurs you can call debug to jump into the Python debugger pdb and examine the problem Alternatively if you call pdb Python will automatically start the debugger on any uncaught exception You can print variables see code execute statements and even walk up and down the call stack to track down the true source of the problem This can be an efficient way to develop and debug code in many cases eliminating the need for print statements or external debugging tools You can also step through a program from the beginning by calling run d theprogram py 4 1 5 History Python stores both the commands you enter and the results it produces You can easily go through previous commands with the up and down arrow keys or access your history in more sophisticated ways Inp
139. to load In general you should consider the notebook as a file meant only to be edited by the Python Notebook app itself not for hand editing Note Binary data such as figures are also saved directly in the JSON file This provides convenient single file portability but means that the files can be large a diff of binary data is also not very meaningful Since the binary blobs are encoded in a single line they affect only one line of the diff output but they are typically very long lines You can use the Cell All Output Clear menu option to remove all output from a notebook prior to committing it to version control if this is a concern The notebook server can also generate a pure Python version of your notebook using the File Download as menu option The resulting py file will contain all the code cells from your notebook verbatim and all Markdown cells prepended with a comment marker The separation between code and Markdown cells is indicated with special comments and there is a header indicating the format version All output is removed when exporting to Python As an example consider a simple notebook called simple ipynb which contains one Markdown cell with the content The simplest notebook one code input cell with the content print Hello IPython and the corresponding output The contents of the notebook document simple ipynb is the following JSON container metadata name simple
140. to_fetch configurables If your machines are on a shared filesystem this step is unnecessary and can be skipped by setting these to empty lists c SSHLauncher to_send c SSHLauncher to_fetch If our default guesses about paths don t work for you or other files should be moved you can manually specify these lists as tuples of local_path remote_path for to_send and remote_path local_path for to_fetch If you do specify these lists explicitly Python will not automatically send connection files so you must include this yourself if they should still be sent retrieved 5 2 4 IPython on EC2 with StarCluster The excellent StarCluster toolkit for managing Amazon EC2 clusters has a plugin which makes deploying Python on EC2 quite simple The starcluster plugin uses ipcluster with the SGE launchers to distribute engines across the EC2 cluster See their ipcluster plugin documentation for more information 5 2 Starting the IPython controller and engines 271 r path to IPython Documentation Release 1 2 1 An Afternoon Hack 5 2 5 Using the ipcontroller and ipengine commands It is also possible to use the ipcontroller and ipengine commands to start your controller and engines This approach gives you full control over all aspects of the startup process Starting the controller and engine on your local machine To use ipcontroller and ipengine to start things on your local machine do the following
141. towards an official Python 1 0 release which will firmly maintain this API compatible for its entire lifecycle There is now an IPython display module that aggregates all display routines and the Python config names pace has all public configuration tools We will continue improving our public API layout so that users only need to import names one level deeper than the main Python package to access all public namespaces IPython notebook file icons The directory docs resources in the source distribution contains SVG and PNG versions of our file icons as well as an Info plist exampl e file with instructions to install them on Mac OSX This is a first draft of our icons and we encourage contributions from users with graphic talent to improve them in the future IPIyl Nb New top level ocate command Add locate entry points these would be useful for quickly locating Python directories and profiles from other non Python applications PR 1762 Examples gt ipython locate Users me ipython gt ipython locate profile foo 86 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Users me ipython profile_foo gt ipython locate profile Users me ipython profile_default gt ipython locate profile dne ProfileLocate Profile u dne not found Other new features and improvements install_ext A new magic function to
142. under default localhost circumstances ari teps 12730 0 12s4424a exec_key a361fe89 92fc 4762 9767 e2f0a05e3130 Togh tem Tlogcat ion LG VO oS If however you are running the controller on a work node on a cluster you will likely need to use ssh tunnels to connect clients from your laptop to it You will also probably need to instruct the controller to listen for engines coming from other work nodes on the cluster An example of ipcontroller client json as created by gt ipcontroller ip ssh login mycluster com url tep 54424 exec_key a361fe89 92fc 4762 9767 e2 0a05e3130 ssh login mycluster com Loeat ton 10 10 0 2 More details of how these JSON files are used are given below A detailed description of the security model and its implementation in Python can be found here Warning Even at its most secure the Controller listens on ports on localhost and every time you make a tunnel you open a localhost port on the connecting machine that points to the Controller If localhost on the Controller s machine or the machine of any client or engine is untrusted then your Controller is insecure There is no way around this with ZeroMQ 5 1 4 Getting Started To use Python for parallel computing you need to start one instance of the controller and one or more instances of the engine Initially it is best to simply start a control
143. user example com l6 cation 192 168 1123 This will be specified if you give the nginessh use example com argument when starting ipcontroller Or you can specify an ssh server on the command line when starting an engine gt ipengine profile foo ssh my login node For example if your system is totally restricted then all connections will actually be loopback and ssh tunnels will be used to connect engines to the controller nodel gt ipcontroller nginessh nodel node2 gt ipengine node3 gt ipcluster engines n 4 Or if you want to start many engines on each node the command ipcluster engines n 4 without any configuration is equivalent to running ipengine 4 times 5 2 Starting the IPython controller and engines 273 IPython Documentation Release 1 2 1 An Afternoon Hack An example using ipcontroller engine with ssh No configuration files are necessary to use ipcontroller engine in an SSH environment without a shared filesystem You simply need to make sure that the controller is listening on an interface visible to the engines and move the connection file from the controller to the engines 1 start the controller listening on an ip address visible to the engine machines controller host TPControllerApp TPControllerApp IPControllerApp TPControllerApp TPControllerApp IPControllerApp TIPContr
144. using the various methods we outline some of the general issues that come up when starting the controller and engines These things come up no matter which method you use to start your Python cluster If you are running engines on multiple machines you will likely need to instruct the controller to listen for connections on an external interface This can be done by specifying the ip argument on the command line or the HubFactory ip configurable in ipcontroller_config py If your machines are on a trusted network you can safely instruct the controller to listen on all interfaces with gt ipcontroller ip Or you can set the same behavior as the default by adding the following line to your ipcontroller_config py c HubFactory ip x c HubFactory location 10 0 1 1 Note ip x instructs ZeroMQ to listen on all interfaces but it does not contain the IP needed for engines clients to know where the controller actually is This can be specified with location 10 0 0 1 the specific IP address of the controller as seen from engines and or clients Python tries to guess this value by default but it will not always guess correctly Check the Location field in your connection files if you are having connection trouble Note Due to the lack of security in ZeroMQ the controller will only listen for connections on localhost by default If you see Timeout errors on engines or clients then the first thing
145. various places Ga l Varoquaux core code ipythonx GUI design discussions etc Lots John Hunter suggestions bug fixes feedback Jorgen Stenarson work on many fronts tests fixes win32 support etc Laurent Dufr chou many improvements to ipython wx standalone app Lukasz Pankowski prefilter edit demo improvements Matt Foster TextMate support in edit Nathaniel Smith fix 237073 Pauli Virtanen fixes and improvements to extensions documentation Prabhu Ramachandran improvements to timeit Robert Kern several extensions Sameer D Costa help on critical bug 269966 Stephan Peijnik feedback on Debian compliance and many man pages Steven Bethard we are now shipping his argparse module Tom Fetherston many improvements to Python demo module Ville Vainio lots of work everywhere features bug fixes etc Vishal Vasta ssh support in ipcluster 2 10 0 10 series 185 IPython Documentation Release 1 2 1 An Afternoon Hack Walter Doerwald work on the Python ipipe system Below we give an overview of new features bug fixes and backwards incompatible changes For a detailed account of every change made feel free to view the project log with bzr log New features New paste magic automatically extracts current contents of clipboard and pastes it directly while correctly handling code that is indented or prepended with gt gt gt or python prompt markers A very usef
146. visited directories and allows you to go to any previously visited one 4 1 Introducing IPython 205 IPython Documentation Release 1 2 1 An Afternoon Hack 4 1 7 Configuration Much of Python can be tweaked through configuration To get started use the command ipython profile create to produce the default config files These will be placed in ipython profile_default or config ipython profile_default and contain comments explaining what the various options do Profiles allow you to use Python for different tasks keeping separate config files and history for each one More details in the profiles section Startup Files If you want some code to be run at the beginning of every Python session the easiest way is to add Python py or IPython ipy scripts to your profile_default startup directory Files here will be executed as soon as the Python shell is constructed before any other code or scripts you have spec ified The files will be run in order of their names so you can control the ordering with prefixes like 10 myimports py 4 2 IPython Tips amp Tricks The Python cookbook details more things you can do with Python 4 2 1 Embed Python in your programs A few lines of code are enough to load a complete Python inside your own programs giving you the ability to work with your data interactively after automatic processing has been completed See the embedding section 4 2 2 Run doctes
147. want to pass these separated with spaces to a command typical for lists if files use the s property Q doc examples 13 gt files 12 fields 8 s Q doc examples 14 gt files lt 14 gt example demo py example gnuplot py extension py seteditoy py setedit Q doc examples 15 gt ls S files example demo py example gnuplot py extension py seteditor py seteditor pyc SLists are inherited from normal python lists so every list method is available Q doc examples 21 gt lines append hey Real world example remove all files outside version control First capture output of hg status Q ipython 28 gt out hg status M IPython extensions ipy_kitcfg py M IPython extensions ipy_rehashdir py build lib IPython Debugger py build lib IPython extensions InterpreterExec py build lib IPython extensions InterpreterPasteInput py lines starting with are not under version control Q ipython 35 gt junk out grep r fields 1 Q ipython 36 gt junk lt 36 gt SList p n 1 s grep fields availab 10 build bdist win32 winexe temp _ctypes py build bdist win32 winexe temp _hashlib py 12 build bdist win32 winexe temp _socket py ji Now we can just remove these files by doing rm junk s The s n p properties The s property returns one string where line
148. with first cell to execute 787 Keyboard selection of multiple lines in the notebook behaves inconsistently 1037 notepad jsonlib TypeError Only whitespace may be used for indentation 970 Default home not writable 7 HOME does not help windows 747 HOMESHARE not a good choice for writable homedir on Windows 810 cleanup utils path get_home_dir 2 Fix the copyright statement in source code files to be accurate 1031 lt esc gt on Firefox crash websocket 684 Store eliminated in configuration and magic commands in 0 11 1026 BUG wrong default parameter in ask_yes_no 880 Better error message if paste fails 1024 autopx magic broken 822 Unicode bug in Itpl when expanding shell variables in syscalls with 1009 Windows regression in cd magic handling of paths 833 Crash python with matplotlib and unequal length arrays 695 Crash handler initialization is too aggressive 1000 Remove duplicates when refilling readline history 2 7 Issues closed in the 0 12 development cycle 147 IPython Documentation Release 1 2 1 An Afternoon Hack 992 Interrupting certain matplotlib operations leaves the inline backend wedged 942 number traits should cast if value doesn t change 1006 Is crashes when run on a UNC path or with non ascii args 944 Decide the default image format for inline figures SVG or PNG 842 Python 3 on Windows pyreadline expected an object with the buffer interface 10
149. your browser You can start it with the command ipython notebook This new interface maintains all the features of Python you are used to as it is a new client that communi cates with the same Python kernels used by the terminal and Qt console But the web notebook provides for a different workflow where you can integrate along with code execution also text mathematical expres sions graphics video and virtually any content that a modern browser is capable of displaying You can save your work sessions as documents that retain all these elements and which can be version controlled emailed to colleagues or saved as HTML or PDF files for printing or publishing statically on the web The internal storage format is a JSON file that can be easily manipulated for manual exporting to other formats 126 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack IPython Notebook Spectrogram Save Idle Notebook P r a Simple spectral analysis L Actions New Open Download ipynb An illustration of the Discrete Fourier Transform N N 1 M Print dri X zae wv k 0 N 1 Cell n 0 Actions Delete using windowing to reveal the frequency content of a sound signal We begin by loading a datafile using SciPy s audio file support Format Code Markdown Output Toggle ClearAll In 1 from scipy io import wavfile Insert Above Below rate x wavfile read home fperez teach py4science book examp
150. 02 ImportError due to incorrect version checking 1001 Ipython source command 954 Python embed doesn t respect namespaces 681 pdb freezes inside qtconsole 698 crash report TypeError can only concatenate list not unicode to list 978 ipython 0 11 buffers external command output till the cmd is done 952 Need user facing warning in the browser if websocket connection fails 988 Error using idlsave 990 ipython notebook kernel dies if matplotlib is not installed 752 Matplotlib figures showed only once in notebook 54 Exception hook should be optional for embedding Python in GUIs 918 Python frontend tests fail without tornado 986 Views created with c direct_view fail 697 Filter out from who names loaded at initialization time 932 Python 0 11 quickref card has superfluous recall and 982 png files with executable permissions 914 Simpler system for running code after InteractiveShell is initialised 911 ipython crashes on startup if readline is missing 971 bookmarks created in 0 11 are corrupt in 0 12 974 object feature tab completion crash 939 ZMQShell always uses default profile 946 Multi tab Close action should offer option to leave all kernels alone 949 Test suite must not require any manual interaction 643 enable gui eventloop integration in ipkernel 965 ipython is crashed without launch python3 2 958 Can t use os X clipboard on with qtconsole 962 Don t require torna
151. 1 PR 3286 PR 3333 PR 3323 PR 3321 PR 3320 PR 3317 PR 3304 PR 3315 PR 3310 PR 3285 PR 3308 PR 3294 workaround spurious CodeMirror scrollbars make setting the notebook dirty flag an event remove long dead zmq frontend py and completer py cull Session digest history Fix get_ipython_dir when HOME is IPEP 13 user expressions and user variables comments in tools gitwash_dumper py changed to Make submodule checks work under Python 3 move anchor link off of heading text start basic tests of ipcluster Launchers allow class __module__ to be None skip submodule check in package managers decode subprocess output in launchers Reenable bracket matching Mpr fixes Use new input transformation API in time magic Organize the JS and less files by component fix test_find_cmd_python catch socket error in utils localinterfaces fix default cluster count don t use get_ipython from builtins in library code notebookapp add missing whitespace to warnings Strip prompts even if the prompt isn t present on the first line Reorganize the python server side of the notebook define __file__ in config files rename file to writefile set unlimited HWM for all relay devices Update Sympy_printing extension load further clarify Image docstring load extensions in builtin trap Speed up AsyncResult _wait_for_outputs 0 fix callbacks as optional in js kernel execute 2 3 Issues closed in the 1
152. 1836 1776 1828 1800 1671 1830 1835 1773 1761 1772 1822 1460 1287 1596 1821 1750 1820 1816 1775 1813 1498 1711 1798 1818 1647 1817 1738 add InlineBackend to ConsoleApp class list Exceptions within multiprocessing crash Ipython notebook kernel Notebook does not exist and permalinks edit magic broken in head resubmitted tasks doesn t have same session name preserve header for resubmitted tasks fix magic menu in qtconsole change default extension to ipy for save r Reintroduce recall _ future__ environments Ismagic lists magics in alphabetical order Use Python import in ipython profile config Update SymPy profile SymPy s latex can now print set and frozenset Edited documentation to use IPYTHONDIR in place of ipython notebook autocomplete fail when typing number aesthetics pass on AsyncResult display_outputs Redirect http to https for notebook Refactor the notebook tab completion tooltip In rename dialog lt return gt should submit ENTER submits the rename notebook dialog Let the user disable random port selection NotebookApp Make the number of ports to retry user configurable Always use filename as the notebook name assert_in not present on Python 2 6 Add assert_in method to nose for Python 2 6 Add tooltip keyboard shortcuts New Tooltip New Completer and JS Refactor a few simple fixes for docs parallel possible bug with latex m
153. 2130 PR 2196 PR 2300 PR 2184 PR 2308 PR 2302 PR 2299 Remove suspicious double quote Added L library search path to cythonmagic cell magic qtconsole Create a prompt newline by inserting a new block w o formatting Fix for 1688 traceback unicode issue use Singleton instance for embed instead of manual global fix missing imports in core interactiveshell remove notification widget leftover Parallel Support get set of nested objects in view e g dv a b Clean up ProgressBar class in example notebook Extra xterm identification in set_term_title Notebook Store the username in a cookie whose name is unique add backport_pr to tools fix names of notebooks for download save make clients use location properly fixes 2361 Refactor notebook templates to use Jinja2 add bash completion example Remove references to version no longer in argparse Github issue 2343 adjust division error message checking to account for Python 3 RemoteError _render_traceback_ calls self render_traceback Normalize line endings for ipexec_validate fix for 2315 Introduce Notification Area Better error messages for common magic commands ENH added StackExchange style MathJax filtering update css for qtconsole in doc adding cluster_id to parallel Client __init__ Add l option to R magic to allow passing in of local namespace Fix for bad command line argument to latex bug fix was crashing when sqlite3 is not ins
154. 2475249 y random random 1073741824 if x x y y lt 1 3486784401L inside 1 10000000000L 25937424601L oldpi pi 61917364224L mypi inside 1 0 nsamples 137858491849L aI endRuf nn arraul Immi d 289254654976L E Test Runs amp Find Results1 E Output Test Results amp Error List Fig 2 6 IPython console embedded in Microsoft Visual Studio from IPython config application import Application logger Application instance log e You can now get help on an object halfway through typing a command For instance typing a zip shows the details of zip It also leaves the command at the next prompt so you can carry on with it e The input history is now written to an SQLite database The API for retrieving items from the history has also been redesigned e The I Python extensions pretty extension has been moved out of quarantine and fully up dated to the new extension API e New magics for loading unloading reloading extensions have been added load_ext Sunload_ext and reload_ext e The configuration system and configuration files are brand new See the configuration system docu mentation for more details e The InteractiveShel 1 class is now a Configurable subclass and has traitlets that determine the defaults and runtime environment The __init__ method has also been refactored so this class can be instantiated and run without the old ipmaker module e The methods of
155. 2659 PR 2363 PR 2662 PR 2328 notebook PR 2668 PR 2663 PR 2656 PR 2312 PR 2571 PR 2641 PR 2638 PR 2301 PR 2616 PR 2572 PR 2574 PR 2582 PR 2591 PR 2526 PR 2559 PR 2546 Doc bugfix user_ns is not an attribute of Magic objects Fix import completion for py3 amp egg files Document OpenMP in cython magic fix jinja2 rendering for password protected notebooks Skip notebook testing if jinja2 is not available Add cython magics to generated documentation Fix pretty print of types when __module__ is not available Fix tox ini Backslashes are misinterpreted as escape sequences by the R interpreter fix error in doc arg gt kwarg and pep 8 for downloads replaced window open with window location assign small bugs in js are fixed Refactor notebook templates to use Jinja2 qtconsole wrap argument list in tooltip to match width of text body addition of classes to generate a link or list of links from files local to the Python HTML pylab_not_importable Catch all exceptions not just RuntimeErrors Fix issue 2660 parsing of help and version arguments Fix irunner tests when PYTHONSTARTUP is set Add bracket matching to code cells in notebook Start to document Javascript undefinied that gt this Fix paste in Python 3 on Mac Ast transfomers Revamp API docs Make Paste Above the default paste behavior Fix 2244 Fix displaying history when output cache is
156. 3 IPython Documentation Release 1 2 1 An Afternoon Hack e keys has been removed but is easily implemented as dview apply lambda globals keys e push_function and push_serialized are removed as push handles functions without issue See also Our Direct Interface doc for a simple tutorial with the DirectView The other major difference is the use of apply When remote work is simply functions the natural return value is the actual Python objects It is no longer the recommended pattern to use stdout as your results due to stream decoupling and the asynchronous nature of how the stdout streams are handled in the new system 5 14 5 Task to LoadBalancedView Load Balancing has changed more than Multiplexing This is because there is no longer a notion of a StringTask or a MapTask there are simply Python functions to call Tasks are now simpler because they are no longer composites of push execute pull clear calls they are a single function that takes arguments and returns objects The load balanced interface is provided by the LoadBalancedView class created by the client In 10 lbview rc load_balanced_view load balancing can also be restricted to a subset of engines In 10 lbview rc load_balanced_view 1 2 3 A simple task would consist of sending some data calling a function on that data plus some data that was resident on the engine already and then pul
157. 32 arg_split Put qtconsole forground when launching allow saving notebook py next to notebook ipynb fix pylab StartMenu item Namespaces for embedding notebook read only disable name field notebook update logo allow customized template and static file paths for the notebook web app BUG Issue 755 qt IPythonWidget execute_file fails if filename contains rename MPIExecLaunchers to MPILaunchers optionally ignore shlex s ValueError in arg_split Shlex unicode Storemagic plugin Add post_install script to create start menu entries in Python 3 Fix tests to work when config ipython contains a symlink Don t transform function calls on IPyAutocall objects protect CRLF from carriage return action Fix for prompts containing newlines Totally remove pager when read only notebook qtconsole allow copy with shortcut in pager fix magics history in two process ipython console Fixing 1112 removing failing asserts for test_carriage_return and test_ e 1089 e 1108 Support carriage return r and beep b characters in the qtconsole Completer usability 2 rebased of pr 1082 2 7 Issues closed in the 0 12 development cycle 137 IPython Documentation Release 1 2 1 An Afternoon Hack 864 Two process terminal frontend ipython core branch e 1082 usability and cross browser compat for completer e 1053 minor improvements to text placement in qtconsole e 1106 Fix display of errors in compiled
158. 321 f N But if your target is actually on a LAN at 192 168 1 123 behind another machine called login then you would do client gt ssh login 12345 192 168 1 16 54321 f N The N on the end are flags that tell ssh to run in the background and don t actually run any commands beyond creating the tunnel See also A short discussion of ssh tunnels http www revsys com writings quicktips ssh tunnel html Stopping Kernels and Consoles Since there can be many consoles per kernel the shutdown mechanism and dialog are probably more com plicated than you are used to Since you don t always want to shutdown a kernel when you close a window you are given the option to just close the console window or also close the Kernel and all other windows Note that this only refers to all other local windows as remote Consoles are not allowed to shutdown the kernel and shutdowns do not close Remote consoles to allow for saving etc Rules e Restarting the kernel automatically clears all Jocal Consoles and prompts remote Consoles about the reset e Shutdown closes all local Consoles and notifies remotes that the Kernel has been shutdown e Remote Consoles may not restart or shutdown the kernel 4 5 7 Qt and the QtConsole An important part of working with the QtConsole when you are writing your own Qt code is to remember that user code in the kernel is not in the same process as the frontend This means that ther
159. 4 e 146 ipythondir using an alternative ipython dir for rc type stuff e 114 Interactive strings get mangled with _ip magic 135 crash on invalid print 69 Usage of mycluster profile in docs and examples 37 Fix colors in output of ResultList on Windows 2 10 0 10 series 2 10 1 Release 0 10 2 Python 0 10 2 was released April 9 2011 This is a minor bugfix release that preserves backward com patibility At this point all Python development resources are focused on the 0 11 series that includes a complete architectural restructuring of the project as well as many new capabilities so this is likely to be the last release of the 0 10 x series We have tried to fix all major bugs in this series so that it remains a viable platform for those not ready yet to transition to the 0 11 and newer codebase since that will require some porting effort as a number of APIs have changed Thus we are not opening a 0 10 3 active development branch yet but if the user community requires new patches and is willing to maintain release such a branch we ll be happy to host it on the Python github repositories Highlights of this release The main one is the closing of github ticket 185 a major regression we had in 0 10 1 where pylab mode with GTK or gthread was not working correctly hence plots were blocking with GTK Since this is the default matplotlib backend on Unix systems this was a major annoyance for many users Man
160. 45 Dec 12 2006 seteditor pyc Now let s filter out the embed lines Q doc examples 4 gt 12 lines grep embed prune 1 Q doc examples 5 gt 12 lt 5 gt SList p n l s grep fields available Value 0 total 23 1 rw rw rw 1 ville None 1163 Sep 30 2006 example demo py 2 rwxrwxrwx 1 ville None 1017 Sep 30 2006 example gnuplot py 3 rwxrwxrwx 1 ville None 339 Jun 11 18 01 extension py 4 rwxrwxrwx 1 ville None 113 Dec 20 2006 seteditor py 5 rwxrwxrwx 1 ville None 245 Dec 12 2006 seteditor pyc Now we want strings having just file names and permissions Q doc examples 6 gt 12 fields 8 0 lt 6 gt SList p n l s grep fields available Value total example demo py rw rw rw example gnuplot py rwxrwxrwx extension py rwxrwxrwx seteditor py rwxrwxrwx seteditor pyc rwxrwxrwx Ope WN EF Note how the line with total does not raise IndexError If you want to split these yielding lists call fields without arguments Q doc examples 7 gt _ fields lt 7 gt total example demo py rw rw rw 232 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack example gnuplot py rwxrwxrwx extension py rwxrwxrwx seteditor py rwxrwxrwx seteditor pyc rwxrwxrwx If you
161. 6 Chapter 5 Using IPython for parallel computing CHAPTER 6 Configuration and customization 6 1 Overview of the IPython configuration system This section describes the Python configuration system The following discussion is for users who want to configure Python to their liking Developers who want to know how they can enable their objects to take advantage of the configuration system should consult the developer guide 6 1 1 The main concepts There are a number of abstractions that the Python configuration system uses Each of these abstractions is represented by a Python class Configuration object Config A configuration object is a simple dictionary like class that holds config uration attributes and sub configuration objects These classes support dotted attribute style access Foo bar in addition to the regular dictionary style access Foo bar Configuration objects are smart They know how to merge themselves with other configuration objects and they automati cally create sub configuration objects Application Application An application is a process that does a specific job The most obvious ap plication is the ipython command line program Each application reads one or more configuration files and a single set of command line options and then produces a master configuration object for the application This configuration object is then passed to the configurable objects that the application creates These c
162. 615 PR 1625 PR 1620 PR 1457 PR 1613 PR 1609 PR 1607 PR 1608 PR 1599 PR 1602 PR 1603 PR 1604 PR 1600 PR 1605 PR 1529 PR 1548 PR 1535 PR 1399 PR 1597 PR 1490 PR 1480 PR 1588 PR 1595 PR 1526 PR 1594 Exclude UserDict when deep reloading NumPy Removed a which shouldn t have been there TST QApplication doesn t quit early enough with PySide evaluate a few dangling validate_message generators clear In prompt numbers on Clear All Output Test the Message Spec Fixes for byte compilation on Python 3 Add show method to figure objects Fix deepreload on Python 3 pyin message now have execution_count Update deepreload to use a rewritten knee py Fixes dreload numpy allow map parallel function for single engine views exit notebook cleanly on SIGINT SIGTERM cleanup sqlitedb temporary db file after tests don t rely on timedelta total_seconds in AsyncResult Fix for run d on Python 3 Fix env magic on Python 3 Remove python3 profile Exclude IPython quarantine from installation Specify encoding for io open in notebook_reformat tests Small fixes for Animation and Progress notebook __all__ feature improvement to dir2 and tests for both add sugar methods properties to AsyncResult Fix pretty printing dispatch Use LaTeX to print various built in types with the SymPy printing extension re enter kernel eventloop after catching SIGINT rename plain
163. 662 save state of qtconsole add history search to qtconsole updated magic_history docstring First version of cluster web service 482 test_run test_tclass fails on Windows 1398 1478 1415 1440 1419 1424 fix tb after SyntaxError key function or lambda in sorted function doesn t find global variables handle exit quit exit quit variants in zmqconsole Fix for failing testsuite when using with xml coverage on windows Add install_ext magic function Win32 shell interactivity Controller should schedule tasks of multiple clients at the same time notebook reset magic fails with StdinNotImplementedError from cherrypy import expose fails when running script form parent directory 2 5 Issues closed in the 0 13 development cycle 119 IPython Documentation Release 1 2 1 An Afternoon Hack 1468 Simplify structure of a Job in the TaskScheduler 875 never build unicode error messages 1107 Tab autocompletion can suggest invalid syntax 1447 1107 Tab autocompletion can suggest invalid syntax 1469 Fix typo in comment insert space 1463 Fix completion when importing modules in the cwd 1437 unfriendly error handling with pythonw and ipython qtconsole 1466 Fix for issue 1437 unfriendly windows qtconsole error handling 1432 Fix ipython directive 1465 allow ipython help subcommand syntax 1394 Wishlist Remove hard dependency on ctypes 1416 Conditional import of ctyp
164. 68 Greedy completer 776 Reworking qtconsole shortcut add fullscreen 790 TST add future unicode_literals test 786 775 redirect_in redirect_out should be constrained to windows only 793 Don t use readline in the ZMQShell 743 Pyglet 774 basic initial mailmap for nice shortlog summaries 770 769 reopened 784 Parse user code to AST using compiler flags 783 always use StringIO never cStringlO 782 flush stdout stderr on displayhook call 622 Make pylab import all configurable 745 Don t assume history requests succeed in qtconsole 725 don t assume cursor selectedText is a string 142 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e 778 e 663 e 762 e 763 722 e 757 e 764 758 e 736 748 e 754 769 753 e 766 e 765 e 741 e 664 e 750 e 433 e 734 e 731 e 730 e 729 T2 e 726 e 705 e 723 don t override execfile on Python 2 Python 3 compatilibility work qtconsole ipython widget s execute_file fails if filename contains spaces or quotes Set context for shortcuts in Console Widget PyPy compatibility ipython el is broken in 0 11 fix colors lt color gt option in py python command args use ROUTER DEALER socket names instead of XREP KREQ enh added authentication ability for webapp Check for tornado before running frontend html tests restore msg_id msg_type aliases in top level of ms
165. 7of20 pi200m ascii 080f20 pi200m ascii 090f20 pi200m ascii 100f20 pi200m ascii 1llof20 pi200m ascii 120f20 pi200m ascii 130f20 pi200m ascii 140f20 pi200m ascii 150f20 download the data files if they don t already exist In 8 v map fetch_pi_file files This is the parallel calculation using the Client map method which applies compute_two_digit_freqs to each file in files in parallel In 9 freqs_all v map compute_two_digit_freqs files Add uP the frequencies from each engine In 10 freqs reduce_freqs freqs_all In 11 plot_two_digit_fregqs freqs Out 11 lt matplotlib image AxesImage object at 0x18beb110 gt In 12 plt title 2 digit counts of 150m digits of pi Out 12 lt matplotlib text Text object at 0x18d1f 9b0 gt The resulting plot generated by Matplotlib is shown below The colors indicate which two digit sequences are more red or less blue likely to occur in the first 150 million digits of pi We clearly see that the sequence 41 is most likely and that 06 and 07 are least likely Further analysis would show that the relative size of the statistical fluctuations have decreased compared to the 10 000 digit calculation 5 11 Parallel examples 325 IPython Documentation Release 1 2 1 An Afternoon Hack 1 498e6 2 digit counts of 150m digits of pi 0 2 4 6 8 0 2 4 6 8 To conclude the
166. 900 3899 3881 2750 python3 install syntax errors OS X 10 8 4 nbconvert to latex fails on notebooks with spaces in file name Temporary Working Directory Test Fails A way to freeze code cells in the notebook 2 3 Issues closed in the 1 0 development cycle 49 IPython Documentation Release 1 2 1 An Afternoon Hack 3893 3823 3885 Resize Local Image Files in Notebook doesn t work nbconvert on windows tex and paths 6699 under Windows ipython3 nbconvert C blabla first_try ipynb to latex write instead of to reference file path in the tex file 3889 3890 3689 3874 3853 3862 test_qt fails due to assertion error qt4 qt double post disregard this issue nbconvert remaining tests Up Down keys don t work to Search previous command history besides Ctrl p Ctrl n CodeMirror locks up in the notebook can only connect to an ipcluster started with v1 0 0 dev master branch using an older ipython v0 13 2 but cannot connect using ipython v1 0 0 dev 3869 2960 3795 3553 3837 3808 2431 3856 3731 3835 3843 3799 3726 3778 3754 3798 3811 3775 3809 3810 3702 3735 custom css not working Keyboard shortcuts ipcontroller process goes to 100 CPU ignores connection requests Ipython and pylab crashes in windows and canopy Cannot set custom mathjax url crash noteb
167. AT bat710 univ lyon1 fr gt New magics King C Shu lt kingshu AT myrealbox com gt Autoindent patch Chris Drexler lt chris AT ac drexler de gt Readline packages for Win32 CygWin Gustavo Cordova Avila lt gcordova AT sismex com gt EvalDict code for nice lightweight string inter polation Kasper Souren lt Kasper Souren AT ircam fr gt Bug reports ideas Gever Tulley lt gever AT helium com gt Code contributions Ralf Schmitt lt ralf AT brainbot com gt Bug reports amp fixes Oliver Sander lt osander AT gmx de gt Bug reports Rod Holland lt rhh AT structurelabs com gt Bug reports and fixes to logging module Daniel Dang Griffith lt pythondev dang AT lazytwinacres net gt Fixes enhancement suggestions for system shell use Viktor Ransmayr lt viktor ransmayr AT t online de gt Tests and reports on Windows installation issues Contributed a true Windows binary installer Mike Salib lt msalib AT mit edu gt Help fixing a subtle bug related to traceback printing W J van der Laan lt gnufnork AT hetdigitalegat nl gt Bash like prompt specials 422 Chapter 9 About IPython IPython Documentation Release 1 2 1 An Afternoon Hack 9 2 Antoon Pardon lt Antoon Pardon AT rece vub ac be gt Critical fix for the multithreaded Python John Hunter lt jdhunter AT nitace bsd uchicago edu gt Matplotlib author helped with all the develop ment of support for matplotlib in Python including making necessary c
168. Bar it doesn t know anything about the othervalue attribute 6 1 5 Configuration file location So where should you put your configuration files Python uses profiles for configuration and by default all profiles will be stored in the so called IPython directory The location of this directory is determined by the following algorithm e If the ipython dir command line flag is given its value is used e If not the value returned by IPython utils path get_ipython_dir is used This func tion will first look at the IPYTHONDIR environment variable and then default to a platform specific default Historical support for the IPYTHON_DIR environment variable will be removed in a future release On posix systems Linux Unix etc Python respects the XDG_CONFIG_HOME part of the XDG Base Directory specification If XDG_CONFIG_HOME is defined and exists XDG_CONFIG_HOME has a default interpretation of SHOME config then Python s config directory will be located in SXDG_CONFIG_HOME ipython If users still have an Python directory in SHOME ipython then that will be used in preference to the system default For most users the default value will simply be something like HOME config ipython on Linux or SHOME ipython elsewhere Once the location of the Python directory has been determined you need to know which profile you are using For users with a single configuration this will simply be de
169. Diagrams of The IPython ZMQ Cluster 415 IPython Documentation Release 1 2 1 An Afternoon Hack Motif Client s Hub Schedulers meee ee eee eee eee ee Se Le oe ee ee ee EEE EEE E E E Engine s Fig 7 7 Engine registration events are published via a PUB socket 416 Chapter 7 IPython developer s guide CHAPTER 8 The IPython API 417 IPython Documentation Release 1 2 1 An Afternoon Hack 418 Chapter 8 The IPython API CHAPTER 9 About IPython 9 1 Credits Python was started and continues to be led by Fernando P rez 9 1 1 Core developers As of this writing core development team consists of the following developers Fernando P rez lt Fernando Perez AT berkeley edu gt Project creator and leader Python core parallel computing infrastructure testing release manager Robert Kern lt rkern AT enthought com gt Co mentored the 2005 Google Summer of Code project work on IPython s core Brian Granger lt ellisonbg AT gmail com gt Parallel computing infrastructure Python core Python notebook Benjamin Min Ragan Kelley lt benjaminrk AT gmail com gt Parallel computing infrastructure Python core Python notebook Ville Vainio lt vivainio AT gmail com gt Python core maintainer of Python trunk from version 0 7 2 to 0 8 4 Gael Varoquaux lt gael varoquaux AT normalesup org gt wxPython Python GUI frontend architec ture Barry Wark lt barrywa
170. Documentation Release 1 2 1 An Afternoon Hack Configuring a cluster profile Next you will need to configure the newly created cluster profile by editing the following configuration files in the cluster directory e ipcluster_config py ipcontroller_config py e ipengine_config py When ipcluster is run these configuration files are used to determine how the engines and controller will be started In most cases you will only have to set a few of the attributes in these files To configure ipcluster to use the Windows HPC job scheduler you will need to edit the following attributes in the file ipcluster_config py Set these at the top of the file to tell ipcluster to use the Windows HPC job scheduler c IPClusterStart controller_launcher_class WindowsHPCControllerLauncher c IPClusterEngines engine_launcher_class WindowsHPCEngineSetLauncher Set these to the host name of the scheduler head node of your cluster WindowsHPCControllerLauncher scheduler HEADNODE c WindowsHPCEngineSetLauncher scheduler HEADNODE Q There are a number of other configuration attributes that can be set but in most cases these will be sufficient to get you started Warning If any of your configuration attributes involve specifying the location of shared directories or files you must make sure that you use UNC paths like host share It is helpful to specify these
171. Fix sphinx template not removing new lines from headers 16 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 4043 PR 4163 PR 4136 PR 4171 PR 4125 PR 4085 PR 4159 PR 4158 PR 4065 PR 4092 PR 4074 PR 4064 PR 4104 PR 4144 PR 4143 PR 4142 PR 4141 PR 4137 PR 4098 PR 4120 PR 4088 PR 4060 PR 4132 PR 4121 PR 4118 PR 4054 PR 4106 PR 4115 PR 4101 PR 4096 PR 4056 PR 4091 don t restore_bytes in from_JSON Fix for incorrect default encoding on Windows catch javascript errors in any output add nbconvert config file when creating profiles Basic exercise of ipython subcommand h and help all nbconvert Fix sphinx preprocessor date format string for Windows don t split cell and div cell CSS generate choices for gui configurable from real mapping do not include specific css in embedable one nbconvert Fix for unicode html headers Windows Python 2 x close Client sockets if connection fails Store default codemirror mode in only 1 place Add way to install MathJax to a particular profile help_end transformer shouldn t pick up in multiline string update example custom js DOC unwrap openssl line in public_server doc add files with a separate add call in backport_pr Restore autorestore option for storemagic pass profile dir instead of profile name to Kernel support input in Pytho
172. Hack In 2 re 0 1 2 Out 2 lt DirectView 0 1 2 gt Other methods for accessing elements such as slicing and negative indexing work by passing the index directly to the client s ids list so negative index In 2 re 1 Out 2 lt DirectView 3 gt or slicing In 3 62 Out 3 lt DirectView 0 2 gt are always the same as In 2 re rce ids 1 Out 2 lt DirectView 3 gt In 3 ref rc ids 2 Out 3 lt DirectView 0 2 gt Also note that the slice is evaluated at the time of construction of the DirectView so the targets will not change over time if engines are added removed from the cluster Execution via DirectView The DirectView is the simplest way to work with one or more engines directly hence the name For instance to get the process ID of all your engines In 5 import os In 6 dview apply_sync os getpid Out 6 1354 1356 1358 1360 Or to see the hostname of the machine they are on In 5 import socket In 6 dview apply_sync socket gethostname Out 6 tesla tesla edison edison edison Note TODO expand on direct execution Data movement via DirectView Since a Python namespace is just a dict Direct View objects provide dictionary style access by key and methods such as get and update for convenience This make the remote namespaces of the engines appear as a local dictionary U
173. Hack Flags Applications can also be passed flags Flags are options that take no arguments They are simply wrappers for setting one or more configurables with predefined values often True False For instance ipcontroller debug is equivalent to ipcontroller Application log_level DEBUG and ipython matploitlib is equivalent to ipython matplotlib auto or ipython no banner is equivalent to ipython TerminallIPythonApp display_banner False dt He UH WH OH HO MH OO Subcommands Some Python applications have subcommands Subcommands are modeled after git and are called with the form command subcommand args Currently the QtConsole is a subcommand of terminal IPython ipython qtconsole profile myprofile and ipcluster is simply a wrapper for its various subcommands start stop engines ipcluster start profile myprofile n 4 To see a list of the available aliases flags and subcommands for an Python application simply pass h or help And to see the full list of configurable options very long pass help all 6 1 8 Design requirements Here are the main requirements we wanted our configuration system to have e Support for hierarchical configuration information e Full integration with command line option parsers Often you want to read a configuration file but then override some of the values with command line options Our configu
174. IPython controller and engines To follow along with this tutorial you will need to start the IPython controller and four IPython engines The simplest way of doing this is to use the ipcluster command 5 5 The IPython task interface 295 IPython Documentation Release 1 2 1 An Afternoon Hack ipcluster start n 4 For more detailed information about starting the controller and engines see our introduction to using Python for parallel computing 5 5 2 Creating a LoadBalancedView instance The first step is to import the Python LPython parallel module and then create a Client instance and we will also be using a LoadBalancedView here called lview In 1 from IPython parallel import Client In 2 re Client This form assumes that the controller was started on localhost with default configuration If not the location of the controller must be given as an argument to the constructor for a visible LAN controller listening on an external port In 2 re Client tcp 192 168 1 16 10101 or to connect with a specific profile you have set up In 3 re Client profile mpi For load balanced execution we will make use of a LoadBalancedView object which can be con structed via the client s load_balanced_view method In 4 lview rc load_balanced_view default load balanced view See also For more information see the in depth explanati
175. MQ devices with ROUTER sockets on either side This allows the queue to relay individual messages to particular targets via zmq IDENTITY routing The Task scheduler may be a MonitoredQueue MQ device in which case the client facing socket is ROUTER and the engine facing socket is DEALER The result of this is that client submitted messages are load balanced via the DEALER socket but the engine s replies to each message go to the requesting client Raw DEALER scheduling is quite primitive and doesn t allow message introspection so there are also Python Schedulers that can be used These Schedulers behave in much the same way as a MonitoredQueue 7 3 Messaging for Parallel Computing 405 IPython Documentation Release 1 2 1 An Afternoon Hack does from the outside but have rich internal logic to determine destinations as well as handle dependency graphs Their sockets are always ROUTER on both sides The Python task schedulers have an additional message type which informs the Hub of the destination of a task as soon as that destination is known Message type task_destination content msg_id abcd 1234 the msg s uuid engine_id 1234 abcd the destination engine s zmq IDENTITY apply In terms of message classes the MUX scheduler and Task scheduler relay the exact same message types Their only difference lies in how the destination is selected The Namespace m
176. P of the Controller If you specified that the HubFactory listen on loopback or all interfaces then Python will try to guess the external IP If you are on a system with VM network devices or many interfaces this guess may be incorrect In these cases you will want to specify the location of the Controller This is the IP of the machine the Controller is on as seen by the clients engines or the SSH server used to tunnel connections 5 2 Starting the IPython controller and engines 275 IPython Documentation Release 1 2 1 An Afternoon Hack For example to set up a cluster with a Controller on a work node using ssh tunnels through the login node an example ipcontroller_config py might contain allow connections on all interfaces from engines engines on the same node will use loopback while engines from other nodes will use an external IP c HubFactory ip a you typically only need to specify the location when there ar xtra interfaces that may not be visible to peer nodes e g VM interfaces c HubFactory location 10 0 1 5 or to get an automatic value try this import socket hostname socket gethostname alternate choices for hostname include socket getfqdn or socket gethostname local x_ip socket gethostbyname_ex hostname 1 1 c HubFactory location ex_ip now instruct clients to use the login node for SSH tunnels c HubFactory ssh_s
177. PGs er ee kee ee eA GS Be er es O23 HISO toe he ak ee We how a he Re od Oe ee i a Ge ee A 9 3 licens and Copyright ss s Sa ek Ee ele EE ee ee ee Ba oe ate Bibliography Python Module Index 259 259 264 278 289 295 302 306 308 310 315 321 327 331 342 347 347 355 358 360 361 363 365 CHAPTER 1 Introduction 1 1 Overview One of Python s most useful features is its interactive interpreter It allows for very fast testing of ideas without the overhead of creating test files as is typical in most programming languages However the interpreter supplied with the standard Python distribution is somewhat limited for extended interactive use The goal of Python is to create a comprehensive environment for interactive and exploratory computing To support this goal Python has three main components e An enhanced interactive Python shell e A decoupled two process communication model which allows for multiple clients to connect to a computation kernel most notably the web based notebook e An architecture for interactive parallel computing All of Python is open source released under the revised BSD license 1 2 Enhanced interactive Python shell IPython s interactive shell ipython has the following goals amongst others 1 Provide an interactive shell superior to Python s default Python has many features for tab completion object introspection system shell access command history re
178. Python will use that instead On Unix platforms including OS X just do S pip install pexpect Note On Python 3 you should actually install pexpect u a unicode safe fork of pexpect Windows users are out of luck as pexpect does not run there 3 5 Dependencies for IPython parallel parallel computing IPython parallel provides a nice architecture for parallel computing with a focus on fluid interactive work flows These features require just one package PyZMQ See the next section for PyYZMQ details On a Unix style platform including OS X if you want to use set uptools you can just do easy_install ipython zmq will include pyzmq Security in Python parallel is provided by SSH tunnels By default Linux and OSX clients will use the shell ssh command but on Windows we also support tunneling with paramiko 3 6 Dependencies for IPython kernel zmq 3 6 1 pyzmq Python 0 11 introduced some new functionality including a two process execution model using ZeroMQ for communication The Python bindings to ZeroMQ are found in the PyZMQ project which is easy_install able once you have ZeroMQ installed If you are on Python 2 6 or 2 7 on OSX or 2 7 on Windows pyzmq has eggs that include ZeroMQ itself IPython kernel zmq depends on pyzmq gt 2 1 4 3 7 Dependencies for the IPython QT console 3 7 1 pyzmq Like the Python parallel package the QT Console requires ZeroMQ and PyZMQ 3 5
179. Several people sharing a single repository If you want to work on some stuff with other people where you are all committing into the same repository or even the same branch then just share it via github First fork ipython into your account as from Making your own copy fork of ipython Then go to your forked repository github page say http github com your user name ipython Click on the Admin button and add anyone else to the repo as a collaborator aF Admin Unwatech iy Pull Request LZ Downl Downloads 0 Wiki 1 Graphs Now all those people can do git clone git githhub com your user name ipython git Remember that links starting with git use the ssh protocol and are read write links starting with git are read only Your collaborators can then commit directly into that repo with the usual git commit am ENH much better code git push origin master pushes directly into your repo Exploring your repository To see a graphical representation of the repository branches and commits gitk all To see a linear list of commits for this branch git log You can also look at the network graph visualizer for your github repo 7 1 Working with IPython source code 381 IPython Documentation Release 1 2 1 An Afternoon Hack 7 1 5 git resources Tutorials and summaries github help has an excellent series of how to guides learn gith
180. Sew ee Ae a 3 5 Dependencies for Python parallel parallel computing 3 6 Dependencies for IPython kernel zmq 2 2 00004 3 7 Dependencies for the Python QT console 4 3 8 Dependencies for the Python HTML notebook 3 9 Dependencies for nbconvert converting notebooks to various formats 4 Using IPython for interactive work Al Introducing IPythoti sot e a sia ede el aS Ee Ee a ee a 4 2 IJPython Tips amp TACKS co 44 4 ae AG oe dba 4a 2d een eR ee ES SG AG Sho 4 3 IPythom TerereneS n e s acem ade a on eee me es ee a a ce eres wea i we A a tee 4 4 I Pythonasasystemshell 2 0 ee 45 AOt Console for Python iea aa Scare i aad wa ae a EG Ae Ra ewe a 4 6 The Python Notebook s 68 4 4 6 ata ka tee ow Le a a a we ee 4 7 Converting notebooks to other formats 0 0 200 02 eee eee ee ee 4 8 Running anotebook server aeie a la e e a e e ee 5 Using Python for parallel computing Sj Overview and getting started socos isani enai a Sp a e e ah ees ta ee ee ee ae h 5 2 Starting the Python controller andengines ss i esoe aca aa a 0 000000 0000 4 5 3 IPython s Dir ct interface s aasa 4d hae AR a we A eM od a 2 es 5 4 Parallel Magic Commands oo 2 2 6 a ea ee ee a es 5 5 The Python task interface s ces ee ee RE ee ee 5 6 Th AsyncResult object gt q diego Baw ea See ee ea SAS eS ad ee eS 5 7 Using MPI with IPython o in acy aoe de daw ae e
181. Sschuh Timothy O Donnell Tom Dimiduk ugurthemaster urielshaolin v923z 2 3 Issues closed in the 1 0 development cycle 23 IPython Documentation Release 1 2 1 An Afternoon Hack e Valentin Haenel e Victor Zverovich e 23 Trevor King y p e Yoav Ram e Zbigniew Jedrzejewski Szmek e Zoltan V r s We closed a total of 1484 issues 793 pull requests and 691 regular issues this is the full list generated with the script tools github_stats py Pull Requests 793 PR 3958 doc update PR 3965 Fix ansi color code for background yellow PR 3964 Fix casing of message PR 3942 Pass on install docs PR 3962 exclude IPython lib kernel in iptest PR 3961 Longpath test fix PR 3905 Remove references to 0 11 and 0 12 from config overview rst PR 3951 nbconvert fixed latex characters not escaped properly in nbconvert PR 3949 log fatal error when PDF conversion fails PR 3947 nbconvert Make writer amp post processor aliases case insensitive PR 3938 Recompile css PR 3948 sphinx and PDF tweaks PR 3943 nbconvert Serve post processor Windows fix PR 3934 nbconvert fix logic of verbose flag in PDF post processor PR 3929 swallow enter event in rename dialog PR 3924 nbconvert Backport fixes PR 3925 Replace pylab flag with matplotlib in usage PR 3910 Added explicit error message for missing configuration arguments PR 3913 grffile to support spaces
182. Switch Tags 1 Branch List 3 Click on the Compare button for your feature branch here mny new feature NAME STATE my new feature Copan Last updated 18 minutes ago 4 If asked select the base and comparison branch names you want to compare Usually these will be master and my new feature where that is your feature branch name 5 At this point you should get a nice summary of the changes Copy the URL for this and post it to the ipython mailing list asking for review The URL will look something like http github com your user name ipython compare master my new feature There s an example at http github com matthew brett nipy compare master find install data See http github com blog 612 introducing github compare view for more detail The generated comparison is between your feature branch my new feature and the place in master from which you branched my new feature In other words you can keep updating master without interfering with the output from the comparison More detail Note the three dots in the URL above master my new feature and see dot2 dot3 7 1 Working with IPython source code 379 IPython Documentation Release 1 2 1 An Afternoon Hack Asking for your changes to be merged with the main repo When you are ready to ask for the merge of your code 1 Goto the URL of your forked repo say http github com your user name ipython git 2 Click on the Pull requ
183. The Balanced queue performs load balancing destination agnostic scheduling It may be a MonitoredQueue de vice but may also be a Python Scheduler that behaves externally in an identical fashion to MQ devices but with additional internal logic stdout err are also propagated from the Engines to the clients via a PUB SUB MonitoredQueue 408 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack Motif Client s Task Contral Balanced Engine s Control Queue ZMQ_FORWARDER Queue Heartbeat Fig 7 1 All the connections involved in connecting one client to one engine 7 4 Connection Diagrams of The IPython ZMQ Cluster 409 IPython Documentation Release 1 2 1 An Afternoon Hack Client s Schedulers p T eee ee eee eee eee ee Se Le oe ee ee ee EEE EEE E E E Registration Engine s Fig 7 2 Engines and Clients only need to know where the Query ROUTER is located to start connecting 410 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack Registration Once a controller is launched the only information needed for connecting clients and or engines is the IP port of the Hub s ROUTER socket called the Registrar This socket handles connections from both clients and engines and replies with the remaining information necessary to establish the remaining connections Clients use this same
184. The principal milestone is the addition of Python nbconvert but there has been a great deal of work improving all parts of Python as well The previous version 0 13 was released on June 30 2012 and in this development cycle we had 12 months of work 700 pull requests merged 600 issues closed non pull requests contributions from 150 authors 4000 commits The amount of work included in this release is so large that we can only cover here the main highlights please see our detailed release statistics for links to every issue and pull request closed on GitHub as well as a full list of individual contributors It includes IPython Documentation Release 1 2 1 An Afternoon Hack Reorganization There have been two major reorganizations in Python 1 0 e Added IPython kernel for all kernel related code This means that TPython zmq has been removed and much of it is now in IPython kernel zmq some of it being in the top level IPython kernel e We have removed the frontend subpackage as it caused unnecessary depth So what was IPython frontend qt is now IPython qt and so on The one difference is that the note book has been further flattened so that TPython frontend html notebook is now just IPython html There is a shim module so Python frontend is still importable in 1 0 but there will be a warning e The Python sphinx directives are now installed in IPython sphinx so they can be importe
185. This allows activation of pylab mode or eventloop integration after starting the kernel which was unavailable in 0 11 Unlike in the terminal this can be set only once and cannot be changed config A new Sconfig magic has been added giving easy access to the Python configuration system at runtime PR 923 Multiline History Multiline readline history has been restored to the Terminal frontend by default PR 838 store The store magic from earlier versions has been updated and re enabled storemagic PR 1029 To autorestore stored variables on startup specify c StoreMagic autorestore True in ipython_config py Major Bugs fixed In this cycle we have closed over 500 issues but a few major ones merit special mention Simple configuration errors should no longer crash Python In 0 11 errors in config files as well as invalid trait values could crash IPython Now such errors are reported and help is displayed Certain SyntaxErrors no longer crash Python e g just typing keywords such as return break etc See 704 IPython path utils such as get_ipython_dir now check for write permissions so Python should function on systems where the default path resolution might point to a read only location such as HOMESHARE on Windows 669 130 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e raw_input now works in the kernel when multiple frontends ar
186. YN DA WN A Install the HPC Pack 2008 on the compute nodes 5 10 Getting started with Windows HPC Server 2008 315 IPython Documentation Release 1 2 1 An Afternoon Hack More details about installing and configuring Windows HPC Server 2008 can be found on the Windows HPC Home Page http www microsoft com hpc Regardless of what steps you follow to set up your cluster the remainder of this document will assume that e There are domain users that can log on to the AD domain and submit jobs to the cluster scheduler e These domain users have shared home directories While shared home directories are not required to use Python they make it much easier to use Python 5 10 3 Installation of IPython and its dependencies Python and all of its dependencies are freely available and open source These packages provide a powerful and cost effective approach to numerical and scientific computing on Windows The following dependencies are needed to run Python on Windows e Python 2 6 or 2 7 http www python org e pywin32 http sourceforge net projects pywin32 e PyReadline https launchpad net pyreadline e pyzmq http github com zeromq pyzmq downloads e Python http ipython org In addition the following dependencies are needed to run the demos described in this document e NumPy and SciPy http www scipy org e matplotlib http matplotlib org The easiest way of obtaining these dependencies is through the Entho
187. _Num self node if isinstance node n int return ast Call func ast Name id Integer ctx ast Load args node keywords return node 6 7 Outdated configuration information that might still be useful Warning All of the information in this file is outdated Until the new configuration system is better documented this material is being kept This section will help you set various things in your environment for your Python sessions to be as efficient as possible All of Python s configuration information along with several example files is stored in a directory named by default HOME config ipython if SHOME config exists Linux or HOME ipython as a secondary default You can change this by defining the environment variable IPYTHONDIR or at runtime with the command line option ipythondir If all goes well the first time you run Python it should automatically create a user copy of the config directory for you based on its builtin defaults You can look at the files it creates to learn more about configuring the system The main file you will modify to configure Python s behavior is called ipythonre with a ini extension under Windows included for reference here This file is very commented and has many variables you can change to suit your taste you can find more details here Here we discuss the basic things you will want to make sure things are working properly from the beginning 6 7 Outdated configur
188. a auto nbconvert Serve from original build directory Add basic logging to nbconvert add missing store_history key to Notebook execute_requests update payload source mention metadata display_data similarity in pyout spec fix incorrect empty docstring Parse markdown correctly when mathjax is disabled 2 3 Issues closed in the 1 0 development cycle 25 IPython Documentation Release 1 2 1 An Afternoon Hack PR 3849 PR 3828 PR 3831 PR 3822 PR 3780 PR 3764 PR 3829 PR 3814 PR 3826 PR 3821 PR 3817 PR 3825 PR 3824 PR 3816 PR 3812 PR 3805 PR 3820 PR 3784 PR 3800 PR 3802 PR 3765 PR 3720 PR 3787 PR 3770 PR 3751 PR 3792 PR 3782 PR 3752 PR 3789 PR 3756 PR 3749 PR 3793 skip a failing test on windows signature_scheme lives in Session update nbconvert doc with new CLI add output flag to nbconvert Added serving the output directory if html based format are selected Cleanup nbconvert templates remove now duplicate this is dev note add Console Widget execute_on_complete_input flag try rtfd add sphinx prolog relax timeouts in terminal console and tests fix more tests that fail when pandoc is missing don t set target on internal markdown links s pylab matplotlib in docs Describe differences between start_ipython and embed Print View has been removed Make it clear that 1 0 is not released yet nbconvert Export flavors amp PDF wr
189. aTeX notation S for inline mathematics and S for displayed mathematics When the Markdown cell is executed the LaTeX portions are automatically rendered in the HTML output as equations with high quality typography This is made possible by MathJax which supports a large subset of LaTeX functionality Standard mathematics environments defined by LaTeX and AMS LaTeX the amsmath package also work such as begin equation end equation and begin align end align New LaTeX macros may be defined using standard methods such as newcommand by placing them anywhere between math delimiters in a Markdown cell These definitions are then available throughout the rest of the Python session See also Markdown Cells example notebook Raw cells Raw cells provide a place in which you can write output directly Raw cells are not evaluated by the notebook When passed through nbconvert raw cells arrive in the destination format unmodified For example this allows you to type full LaTeX into a raw cell which will only be rendered by LaTeX after conversion by nbconvert Heading cells You can provide a conceptual structure for your computational document as a whole using different levels of headings there are 6 levels available from level 1 top level down to level 6 paragraph These can be used later for constructing tables of contents etc As with Markdown cells a heading cell is replaced by a rich text rendering of the hea
190. ad directories Simple class CM configurable Take 2 Updates storemagic extension to allow for specifying variable name to load backport If Modified Since fix from tornado IFrame VimeoVideo ScribdDocument Fix small inconsistency in nbconvert etype gt ename Fix issue 2563 core profiledir check_startup_dir doesn t work inside py2exe d instal Fix inheritance_diagram Sphinx extension for Sphinx 1 2 Update link to extensions index Separate InputSplitter for transforming whole cells Improve completer finish up PR 3116 Add new keycodes Key the root modules cache by sys path entries clarify file docstring BUG Fix the set and frozenset pretty printer to handle the empty case correctly better UsageError for cell magic with no body 2 3 Issues closed in the 1 0 development cycle 35 IPython Documentation Release 1 2 1 An Afternoon Hack PR 3184 PR 3175 PR 3173 PR 2635 PR 3172 PR 3170 PR 3159 PR 3105 PR 3144 PR 3141 PR 3137 PR 3072 PR 3147 PR 3152 PR 3162 PR 2981 PR 3058 PR 3041 PR 2447 PR 3108 PR 2791 PR 3157 PR 3148 PR 3143 PR 3136 PR 3126 PR 3135 PR 2889 PR 3068 PR 3133 PR 3102 PR 3064 Cython cache Added missing s Little bits of documentation cleanup Improve Windows start menu shortcuts 2 Add missing import in Python parallel magics example default application logger shouldn t propagate Auto
191. ad doctest anymore as this was causing many problems with the test suite It still resets doctest master to None While we have not deliberately broken Python 2 4 compatibility only minor testing was done with Python 2 4 while 2 5 and 2 6 were fully tested But if you encounter problems with 2 4 please do report them as bugs The ipcluster now requires a mode argument for example to start a cluster on the local machine with 4 engines you must now type ipcluster local n 4 The controller now has a r flag that needs to be used if you want to reuse existing furl files Other wise they are deleted the default Remove ipy_leo py You can use easy_install ipython extension to get it done to decouple it from ipython release cycle 2 11 0 9 series 2 11 1 Release 0 9 1 This release was quickly made to restore compatibility with Python 2 4 which version 0 9 accidentally broke No new features were introduced other than some additional testing support for internal use 2 11 2 Release 0 9 New features e All furl files and security certificates are now put in a read only directory named ipython security e A single function get_ipython_dir in IPython genutils that determines the user s Python directory in a robust manner 188 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Laurent s WX application has been given a top level script called ipython wx
192. agic recall under pypy 1047 Document the notebook format 102 Properties accessed twice for classes defined interactively 16 Ystore raises exception when storing compiled regex 67 tab expansion should only take one directory level at the time 62 Global variables undefined in interactive use of embedded ipython shell 57 debugging with ipython does not work well outside ipython 38 Line entry edge case error 980 Update parallel docs for new parallel architecture 1017 Add small example about ipcluster ssh startup 1041 Proxy Issues 146 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 967 KernelManagers don t use zmq eventloop properly 1055 All Magics display on Ubuntu 1054 ipython explodes on syntax error 1051 ipython3 set_next_input failure 693 run i no longer works after reset in terminal 29 cPickle works in standard interpreter but not in Python e 1050 ipython3 broken by commit 8bb887c8c2c447bf7 e 1048 Update docs on notebook password e 1046 Searies of questions issues e 1045 crash when exiting previously launched embedded sub shell e 1043 pylab doesn t work in qtconsole e 1044 run p doesn t work in python 3 e 1010 emacs freezes when ipython complete is called 82 Update devel docs with discussion about good changelogs 116 Update release management scipts and release revision for git 1022 Pylab banner shows up
193. ailure on engine resubmitted uuid ascii msg_id of resubmitted task if applicable result_header dict header for result result_content dict content for result result_buffers list bytes buffers containing serialized request objects queue str The name of the queue for the task mux or task pyin str Python input source pyout dict Python output pyout message content pyerr dict Python traceback pyerr message content stdout str Stream of stdout data stderr str Stream of stderr data MongoDB operators we emulate on all backends Operator Python equivalent in in nin not in eq ne l ge gt gte gt le lt lte lt The DB Query is useful for two primary cases 5 8 IPython s Task Database 309 IPython Documentation Release 1 2 1 An Afternoon Hack 1 deep polling of task status or metadata 2 selecting a subset of tasks on which to perform a later operation e g wait on result purge records resubmit 5 8 3 Example Queries To get all msg_ids that are not completed only retrieving their ID and start time In 1 incomplete rc db_query completed None keys msg_id started All jobs started in the last hour by me In 1 from datetime import datetime timedelta In 2 hourago datetime now timedelta 1 24 In 3
194. al work will not be clobbered by a system shell history for example Autoindent IPython can recognize lines ending in and indent the next line while also un indenting automatically after raise or return This feature uses the readline library so it will honor your inputrc configuration or whatever file your INPUTRC variable points to Adding the following lines to your inputrc file can make indent ing unindenting more convenient M i indents M u unindents if you don t already have a inputre file you need this include Sinclude etc inputre Sif Python Ww M i M u d d d d endif Note that there are 4 spaces between the quote marks after M i above 214 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack Warning Autoindent is ON by default but it can cause problems with the pasting of multi line indented code the pasted code gets re indented on each line A magic function autoindent allows you to toggle it on off at runtime You can also disable it permanently on in your ipython_config py file set TerminalInteractiveShell autoindent False If you want to paste multiple lines in the terminal it is recommended that you use Spaste Customizing readline behavior All these features are based on the GNU readline library which has an extremely customizable interface Normally readline is
195. all the lines in it thus restoring the state of a previous session This feature is not quite perfect but can still be useful in many cases The log files can also be used as a way to have a permanent record of any code you wrote while experiment ing Log files are regular text files which you can later open in your favorite text editor to extract code or to clean them up before using them to replay a session The logstart function for activating logging in mid session is used as follows Slogstart log_name log_mode If no name is given it defaults to a file named ipython_log py in your current working directory in rotate mode see below logstart name saves to file name in backup mode It saves your history up to that point and then continues logging 4 3 IPython reference 215 IPython Documentation Release 1 2 1 An Afternoon Hack logstart takes a second optional parameter logging mode This can be one of note that the modes are given unquoted e over overwrite existing log_name e backup rename if exists to log_name and start log_name e append well that says it e rotate create rotating logs log_name 1 log_name 2 etc The logoff and logon functions allow you to temporarily stop and resume logging to a file which had previously been started with logstart They will fail with an explanation if you try to use them before logging has been
196. allel job on your cluster and then have a remote collaborator connect to it and pull back data into their local Python session for plotting and analysis Run a set of tasks on a set of CPUs using dynamic load balancing Tip At the SciPy 2011 conference in Austin Min Ragan Kelley presented a complete 4 hour tutorial on the use of these features and all the materials for the tutorial are now available online That tutorial provides an excellent hands on oriented complement to the reference documentation presented here 5 1 3 Architecture overview The Python architecture consists of four components e The Python engine e The IPython hub e The Python schedulers e The controller client These components live in the ILPython parallel package and are installed with I Python They do however have additional dependencies that must be installed For more information see our installation documentation IPython engine The Python engine is a Python instance that takes Python commands over a network connection Eventually the Python engine will be a full Python interpreter but for now it is a regular Python interpreter The engine can also handle incoming and outgoing Python objects sent over a network connection When multiple engines are started parallel and distributed computing becomes possible An important feature of an Python engine is that it blocks while user code is being executed Read on for how the Python control
197. ample notebook Basic Output example notebook a warning about doing roundtrip conversions Keyboard shortcuts All actions in the notebook can be performed with the mouse but keyboard shortcuts are also available for the most common ones The essential shortcuts to remember are the following e Shift Enter run cell Execute the current cell show output if any and jump to the next cell below If Shift Enter is invoked on the last cell a new code cell will also be created Note that in the notebook typing Enter on its own never forces execution but rather just inserts a new line in the current cell Shift Enter is equivalent to clicking the Cell Run menu item e Ctrl Enter run cell in place Execute the current cell as if it were in terminal mode where any output is shown but the cursor remains in the current cell The cell s entire contents are selected after execution so you can just start typing and only the new input will be in the cell This is convenient for doing quick experiments in place or for querying things like filesystem content without needing to create additional cells that you may not want to be saved in the notebook e Alt Enter run cell insert below Executes the current cell shows the output and inserts a new cell between the current cell and the cell below if one exists This is thus a shortcut for the sequence Shift Enter Ctrl m a Ctrl m a adds anew cell above the current one
198. amples tests embed 1171 Recognise virtualenvs 1479 test_extension failing in Windows 1633 Fix installing extension from local file on Windows 1644 Update copyright date to 2012 1636 Test_deepreload breaks pylab irunner tests 1645 Exclude UserDict when deep reloading NumPy 1454 make it possible to start engine in disabled mode and enable later 1641 Escape code for the current time in PromptManager 1638 ipython console clobbers custom sys path 2 5 Issues closed in the 0 13 development cycle 115 IPython Documentation Release 1 2 1 An Afternoon Hack 1637 1536 1628 1631 1629 1621 1627 1470 1624 1612 1615 1622 1625 1626 1623 1619 1620 Removed a which shouldn t have been there ipython 0 12 embed shell won t run startup scripts error QApplication already exists in TestKillRing TST QApplication doesn t quit early enough with PySide evaluate a few dangling validate_message generators clear In prompt numbers on Clear All Output Test the Message Spec SyntaxError on setup py install with Python 3 Fixes for byte compilation on Python 3 pylab inline fig show non existent in notebook Add show method to figure objects deepreload fails on Python 3 Fix deepreload on Python 3 Failure in new dreload tests under Python 3 2 iPython matplotlib Memory error with imshow pyin messages should have execution_count pyin mes
199. an be zero to many Clients connected to a Kernel at any given time The KernelManager now automatically restarts the kernel when it dies rather than requiring user input at the notebook or QtConsole UI which may or may not exist at restart time In process kernels The Python language frontends particularly the Qt console may now communicate with in process kernels in addition to the traditional out of process kernels An in process kernel permits direct access to the kernel namespace which is necessary in some applications It should be understood however that the in process kernel is not robust to bad user input and will block the main GUD thread while executing Developers must decide on a case by case basis whether this tradeoff is appropriate for their application Parallel IPython parallel has had some refactoring as well There are many improvements and fixes but these are the major changes Connections have been simplified All ports and the serialization in use are written to the connection file rather than the initial two stage system Serialization has been rewritten fixing many bugs and dramatically improving performance serializ ing large containers Load balancing scheduler performance with large numbers of tasks has been dramatically improved There should be fewer hopefully zero false positives for engine failures Increased compatibility with various use cases that produced serialization argument errors
200. ance for your profile and use the resulting instance to do a simple interactive parallel computation In the code and screenshot that follows we take a simple Python function and apply it to each element of an array of integers in parallel using the DirectView map method In 1 from IPython parallel import x In 2 c Client profile mycluster In 3 view c In 4 c ids Out 4s FO dy 2y 3 45 5 Or Ty Gr 97 BO Liy dae 13 14 320 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack In 5 def f x return xx 10 In 6 view map f range 15 f is applied in parallel Out 6 0 1 1024 59049 1048576 9765625 60466176 282475249 1073741824 3486784401L 10000000000L 25937424601L 61917364224L 137858491849L 289254654976L i The map method has the same signature as Python s builtin map function but runs the calculation in parallel More involved examples of using Direct View are provided in the examples that follow 5 11 Parallel examples In this section we describe two more involved examples of using an Python cluster to perform a parallel computation We will be doing some plotting so we start Python with matplotlib integration by typing ipython matplotlib at the system command line Or you can enable matplotlib integration at any point with
201. and it has received numerous fixes We expect this code to be architecturally better integrated with Gael s WX ipython widget over the next few releases The Editor synchronization work by Vivian De Smedt has been merged in This code adds a number of new editor hooks to synchronize with editors under Windows A new still experimental but highly functional WX shell by Gael Varoquaux This work was spon sored by Enthought and while it s still very new it is based on a more cleanly organized arhictecture of the various Python components We will continue to develop this over the next few releases as a model for GUI components that use Python Another GUI frontend Cocoa based Cocoa is the OSX native GUI framework authored by Barry Wark Currently the WX and the Cocoa ones have slightly different internal organizations but the whole team is working on finding what the right abstraction points are for a unified codebase As part of the frontend work Barry Wark also implemented an experimental event notification system that various ipython components can use In the next release the implications and use patterns of this system regarding the various GUI options will be worked out Python finally has a full test system that can test docstrings with Python specific functionality There are still a few pieces missing for it to be widely accessible to all users so they can run the test suite at any time and report problems but it n
202. are put into the IPYTHONDIR profile_default security directory of the client s host they will be found automatically Otherwise the full path to them has to be passed to the client s constructor 5 2 2 Using ipcluster The ipcluster command provides a simple way of starting a controller and engines in the following situations 1 When the controller and engines are all run on localhost This is useful for testing or running on a multicore computer 2 When engines are started using the mpiexec command that comes with most MPI MPI implemen tations 3 When engines are started using the PBS PBS batch system or other gsub systems such as SGE 4 When the controller is started on localhost and the engines are started on remote nodes using ssh 5 When engines are started using the Windows HPC Server batch system Note Currently ipcluster requires that the IPYTHONDIR profile_ lt name gt security direc tory live on a shared filesystem that is seen by both the controller and engines If you don t have a shared file system you will need to use ipcont roller and ipengine directly Under the hood ipcluster just uses ipcontroller and ipengine to perform the steps described above The simplest way to use ipcluster requires no configuration and will launch a controller and a number of engines on the local machine For instance to start one controller and 4 engines on localhost just do ipcluster start n 4
203. arkdown Aborted parallel tasks can t be resubmitted Change behavior of ipython notebook port IPython embed_kernel issues 110 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 1610 1576 1515 1812 1811 1809 1810 1808 1742 1802 1062 1807 1805 1803 1804 1492 1562 1739 1770 1532 1707 1797 1796 1795 1788 1793 1794 1777 1792 1791 1787 1785 Basic bold and italic in HTML output cells Start and stop kernels from the notebook dashboard impossible to shutdown notebook kernels Ensure AsyncResult display_outputs doesn t display empty streams warn on nonexistent exclusions in iptest test suite error in Python zmq on windows fix for 1809 failing tests in Python zmq Reposition alternate upload for firefox need cross browser OS language test Check for custom_exceptions only once cythonmagic tests should be skipped if Cython not available warning message in Python extensions test add missing cython exclusion in iptest Fixed a vevarsall bat error on win32 Py2 7 when trying to compile with m MPI parallel px bug Fixed a vevarsall bat error on win32 Py2 7 when trying to compile with mingw Drag target very small if Python Dashboard has no notebooks Offer a method other than drag n drop to upload notebooks Dashboard improvement necessary merge of 1658 and 1676 fix 1492 Cyth
204. art and stop the controller and engines any many times as you want in the future just make sure to tell the controller to reuse the file Note You may ask the question what ports does the controller listen on if you don t tell is to use specific ones The default is to use high random port numbers We do this for two reasons i to increase security through obscurity and ii to multiple controllers on a given host to start and automatically use different ports Log files All of the components of IPython have log files associated with them These log files can be extremely useful in debugging problems with IPython and can be found in the directory IPYTHONDIR profile_ lt name gt log Sending the log files to us will often help us to debug any problems Configuring ipcontroller The IPython Controller takes its configuration from the file ipcontroller_config py in the active profile directory Ports and addresses In many cases you will want to configure the Controller s network identity By default the Controller listens only on loopback which is the most secure but often impractical To instruct the controller to listen on a specific interface you can set the HubFactory ip trait To listen on all interfaces simply specify c HubFactory ip x When connecting to a Controller that is listening on loopback or behind a firewall it may be necessary to specify an SSH server to use for tunnels and the external I
205. ary file and it returns the contents of your editing session as a string variable Thanks to I Python s output caching mechanism this is automatically stored In 1 edit IPython will make a temporary file named tmp ipython_edit_yR HCN py Editing done Executing edited code hello this is a temporary file Out 1 print hello this is a temporary file n Now if you call Sedit p IPython tries to open an editor with the same data as the last time you used edit So if you haven t used edit in the meantime this same contents will reopen however it will be done in a new file This means that if you make changes and you later want to find an old version you can always retrieve it by using its output number via edit _NN where NN is the number of the output prompt Continuing with the example above this should illustrate this idea In 2 edit p IPython will make a temporary file named tmp ipython_edit_nA09Qk py Editing done Executing edited code hello now I made some changes Out 2 print hello now I made some changes n In 3 edit _1 IPython will make a temporary file named tmp ipython_edit_gy6 zD py Editing done Executing edited code hello this is a temporary file IPython version control at work Out 3 print hello this is a temporary file nprint IPython version conty 4 2 IPytho
206. asks Resubmit Sometimes you may want to re run a task This could be because it failed for some reason and you have fixed the error or because you want to restore the cluster to an interrupted state For this the Client has arc resubmit method This simply takes one or more msg_ids and returns an AsyncHubResult for the result s You cannot resubmit a task that is pending only those that have finished either successful or unsuccessful 5 5 6 Schedulers There are a variety of valid ways to determine where jobs should be assigned in a load balancing situation In Python we support several standard schemes and even make it easy to define your own The scheme can be selected via the scheme argument to ipcontroller or in the TaskScheduler schemename attribute of a controller config object The built in routing schemes To select one of these schemes simply do ipcontroller scheme lt schemename gt for instance ipcontroller scheme lru lru Least Recently Used Always assign work to the least recently used engine A close relative of round robin it will be fair with respect to the number of tasks agnostic with respect to runtime of each task plainrandom Plain Random Randomly picks an engine on which to run 300 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack twobin Two Bin Random Requires numpy Pick two engines at rando
207. at all They receive the whole block as a single string As a line magic example the cd magic works just like the OS command of the same name In 8 cd home fperez The following uses the builtin t imeit in cell mode In 10 timeit x range 10000 min x max x 1000 loops best of 3 438 us per loop 4 3 IPython reference 209 IPython Documentation Release 1 2 1 An Afternoon Hack In this case x range 10000 is called as the line argument and the block with min x and max x is called as the cell body The timeit magic receives both If you have automagic enabled as it by default you don t need to type in the single explicitly for line magics Python will scan its internal list of magic functions and call one if it exists With automagic on you can then just type cd mydir to go to directory mydir In 9 cd mydir home fperez mydir Note that cell magics always require an explicit 3 prefix automagic calling only works for line magics The automagic system has the lowest possible precedence in name searches so defining an identifier with the same name as an existing magic function will shadow it for automagic use You can still access the shadowed magic function by explicitly using the character at the beginning of the line An example with automagic on should clarify all this In 1 cd ipython cd is called by automagic hom
208. at argparse has become part of Python 2 7 this will be less of an issue but Steven s relicensing allowed us to start updating Python to using argparse well before Python 2 7 Many thanks Robustness improvements so that Python doesn t crash if the readline library is absent though obvi ously a lot of functionality that requires readline will not be available Improvements to tab completion in Emacs with Python 2 6 Logging now supports timestamps see logst art for full details A long standing and quite annoying bug where parentheses would be added to print statements under Python 2 5 and 2 6 was finally fixed Improved handling of libreadline on Apple OSX Fix reload method of Python demos which was broken Fixes for the ipipe ibrowse system on OSX Fixes for Zope profile Fix Ytimeit reporting when the time is longer than 1000s Avoid lockups with or in SunOS due to a bug in termios The usual assortment of miscellaneous bug fixes and small improvements The following people contributed to this release please let us know if we omitted your name and we ll gladly fix this in the notes for the future Beni Cherniavsky Boyd Waters David Warde Farley Fernando Perez G khan Sever John Hunter Justin Riley Kiorky Laurent Dufrechou Mark E Smith Matthieu Brucher Satrajit Ghosh Sebastian Busch Vaclav Smilauer 184 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Aftern
209. ate websockets will fail e The diigo Chrome extension http help diigo com tools chrome extension seems to interfere with scrolling There are some early reports that the Notebook works on Internet Explorer 10 but we expect there will be some CSS issues related to the flexible box model 3 8 Dependencies for the IPython HTML notebook 201 IPython Documentation Release 1 2 1 An Afternoon Hack 3 9 Dependencies for nbconvert converting notebooks to various formats 3 9 1 pandoc The most important dependency of nbconvert is Pandoc a document format translation program This is not a Python package so it cannot be expressed as a regular Python dependency with setuptools To install pandoc on Linux you can generally use your package manager sudo apt get install pandoc On other platforms you can get pandoc from their website 202 Chapter 3 Installation CHAPTER 4 Using IPython for interactive work 4 1 Introducing IPython You don t need to know anything beyond Python to start using Python just type commands as you would at the standard Python prompt But Python can do much more than the standard prompt Some key features are described here For more information check the tips page or look at examples in the Python cookbook If you ve never used Python before you might want to look at the official tutorial or an alternative Dive into Python 4 1 1 The four most helpful co
210. atform Qt windows created within imported functions won t show qtconsole ignores exec_lines segfault in qtconsole when kernel quits Failure error nothing to repeat Ipython gtconsole pylab config issue RuntimeError in completer run filename autocompletion The kernel heartbeat has been inactive error log_level is broken in the ipython Application windows source install doesn t create scripts correctly Make sure that the Windows installer handles the top level Python scripts Two displayhooks in zmq Make sure that the Windows HPC scheduler support is working for 0 11 98 Should be able to get help on an object mid command 174 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 529 476 175 156 470 30 503 456 451 79 522 e 524 523 e 519 e 516 e 520 450 e 518 e 178 e 515 e 510 508 499 426 e 497 493 489 458 488 unicode problem in qtconsole for windows Separate input area in Qt Console Qt console needs configuration support Key history lost when debugging program crash decorator uses deprecated features readline in OS X does not have correct key bindings merge IPython parallel streamsession and Python zmq session pathname in document punctuated by dots not slashes Allow switching the default image format for inline mpl backend Implement more robust handling of conf
211. ation Release 1 2 1 An Afternoon Hack PR 5163 Simplify implementation of Temporary WorkingDirectory PR 5105 add index to format to support py2 6 1 2 0 PR 4972 Work around problem in doctest discovery in Python 3 4 with PyQt PR 4934 ipython profile create respects ipython dir PR 4845 Add Origin Checking PR 4928 use importlib machinery when available PR 4849 Various unicode fixes mostly on Windows PR 4880 set profile name from profile_dir PR 4908 detect builtin docstrings in oinspect PR 4909 sort dictionary keys before comparison ordering is not guaranteed PR 4903 use https for all embeds PR 4868 Static path fixes PR 4820 fix regex for cleaning old logs with ipcluster PR 4840 Error in Session send_raw PR 4762 whitelist alphanumeric characters for cookie_name PR 4748 fix race condition in profiledir creation PR 4720 never use ssh multiplexer in tunnels PR 4738 don t inject help into user_ns PR 4722 allow purging local results as long as they are not outstanding PR 4668 Make non ASCII docstring unicode PR 4639 Minor import fix to get qtconsole with pylab qt working PR 4453 Play nice with App Nap PR 4609 Fix bytes regex for Python 3 PR 4488 fix typo in message spec doc PR 4346 getpass on Windows amp Python 2 needs bytes prompt PR 4230 Switch correctly to the user s default matplotlib backend after inline PR 4214 engine ID m
212. ation information that might still be useful 365 IPython Documentation Release 1 2 1 An Afternoon Hack 6 7 1 Color The default Python configuration has most bells and whistles turned on they re pretty safe But there s one that may cause problems on some systems the use of color on screen for displaying information This is very useful since Python can show prompts and exception tracebacks with various colors display syntax highlighted source code and in general make it easier to visually parse information The following terminals seem to handle the color sequences fine e Linux main text console KDE Konsole Gnome Terminal E term rxvt xterm e CDE terminal tested under Solaris This one boldfaces light colors e X Emacs buffers See the emacs_ section for more details on using Python with X Emacs e A Windows XP 2k command prompt with pyreadline e A Windows XP 2k CygWin shell Although some users have reported problems it is not clear whether there is an issue for everyone or only under specific configurations If you have full color support under cygwin please post to the Python mailing list so this issue can be resolved for all users These have shown problems e Windows command prompt in WinXP 2k logged into a Linux machine via telnet or ssh e Windows native command prompt in WinXP 2k without Gary Bishop s extensions Once Gary s readline library is installed the normal WinXP 2k command prompt
213. bar on tooltip in Chrome 24 on Ubuntu 12 04 ipdb completely broken in Qt console Document the meaning of milestone issues tags for users Do not auto show tooltip if docstring empty Web page No link to front page from documentation nbconvert to slideshow misses some of the math in markdown cells increase and make timeout configurable in console completion ipcluster only running on one cpu better message for unsupported nbformat Can not stop ipython kernel on windows Need direct entry point to given notebook ImportError cannot import name check_linecache_ipython nbconvert Final output file should be in same directory as input file history o works but history with n produces identical results error when attempting to run make in docs directory ipython nbconvert crashes with ValueError Invalid format string nbconvert unhelpful error when pandoc isn t installed markdown cell cursor misaligned in notebook mutiple input fields for debug in the notebook after resetting the kernel PyCharm has problems with Python working inside PyPy created by virtualenv Code completion Complete on dictionary keys pylab and matplotlib flag nbconvert Unicode error with minus sign nbconvert does not process SVGs into PDFs nbconvert figures not extracting with Python 3 x note new dependencies in docs setup py pagedown do not target_blank anchor link bad message when pylab fails due import other than matplotlib ipyt
214. bconvert Fix app tests on Window7 w Python 3 3 PR 3978 fix existing with non localhost IP PR 3939 minor checkpoint cleanup PR 3981 BF fix nbconvert rst input prompt spacing PR 3960 Don t make sphinx a dependency for importing nbconvert PR 3973 logging Formatter is not new style in 2 6 Issues 18 4024 4095 4156 3818 4134 3998 4078 3990 4015 4059 nbconvert markdown issues Catch js error in append html in stream pyerr Specifying gui tk at the command line nbconvert can t handle Heading with Chinese characters on Japanese Windows OS multi line parser fails on in comment qtconsole and notebook sample custom js needs to be updated StoreMagic autorestore not working in 1 0 0 Buitlin input doesn t work over zmq nbconvert fails to convert all the content of a notebook Issues with Ellipsis literal in Python 3 18 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 4103 4100 4005 4020 3945 3977 3980 3968 Wrong default argument of Direct View clear parallel client client references undefined error EngineError IPython start_kernel doesn t work Python parallel map fails on numpy arrays nbconvert commandline tests fail Win7x64 Py3 3 unable to complete remote connections for two process nbconvert rst output lacks needed blank lines TypeError super argument 1 must be type not classobj
215. bconvert is a level preview code in 1 0 To use nbconvert to convert various file formats ipython nbconvert to html ipynb See ipython nbconvert help for more information nbconvert depends on pandoc for many of the translations to and from various formats See also Converting notebooks to other formats Notebook Major changes to the Python Notebook in 1 0 The notebook is now autosaved by default at an interval of two minutes When you press save or Ctrl S a checkpoint is made in a hidden folder This checkpoint can be restored so that the autosave model is strictly safer than traditional save If you change nothing about your save habits you will always have a checkpoint that you have written and an autosaved file that is kept up to date The notebook supports raw_input input and thus also debug and many other Python calls that expect user input You can load custom javascript and CSS in the notebook by editing the files ipython locate profile static custom custom js css Add html 3svg javascript and latex cell magics for writing raw output in note book cells add a redirect handler and anchors on heading cells so you can link across notebooks directly to heading cells in other notebooks Images support width and height metadata and thereby 2x scaling retina support _repr_foo_ methods can return a tuple of data metadata where metadata is a dict containing metadata
216. bconvert will have the same base name as the notebook and will be placed in the current working directory Any supporting files graphics etc will be placed in a new directory with the same base name as the notebook suffixed with _files ipython nbconvert notebook ipynb ls notebook ipynb notebook html notebook_files For simple single file output such as html markdown etc the output may be sent to standard output with ipython nbconvert to markdown notebook ipynb stdout Multiple notebooks can be specified from the command line ipython nbconvert notebook ipynb ipython nbconvert notebookl ipynb notebook2 ipynb or via a list in a configuration file say mycfg py containing the text c get_config c NbConvertApp notebooks notebookl ipynb notebook2 ipynb and using the command ipython nbconvert config mycfg py 4 7 1 Notebook JSON file format Notebook documents are JSON files with an ipynb extension formatted as legibly as possible with minimal extra indentation and cell content broken across lines to make them reasonably friendly to use 4 7 Converting notebooks to other formats 253 IPython Documentation Release 1 2 1 An Afternoon Hack in version control workflows You should be very careful if you ever manually edit this JSON data as it is extremely easy to corrupt its internal structure and make the file impossible
217. be 1 0 ar progress len ar e AsyncResult wait_interactive will wait for the result to finish but print out status up dates on progress and elapsed time while it waits For use after the tasks are done e ar serial_time is the sum of the computation time of all of the tasks done in parallel e ar wall_time is the time between the first task submitted and last result received This is the actual cost of computation including Python overhead 5 6 The AsyncResult object 303 IPython Documentation Release 1 2 1 An Afternoon Hack Note wall_time is only precise if the Client is waiting for results when the task finished because the received timestamp is made when the result is unpacked by the Client triggered by the spin call If you are doing work in the Client and not waiting spinning then received might be artificially high An often interesting metric is the time it actually cost to do the work in parallel relative to the serial compu tation and this can be given simply with speedup ar serial_time ar wall_time 5 6 3 Map results are iterable When an AsyncResult object has multiple results e g the AsyncMapResult object you can actually iterate through results themselves and act on them as they arrive from __future_ import print_function import time from IPython import parallel create client amp view re parallel Client dv rc v rce
218. before GUI support is provided without the use of any threads by directly integrating GUI event loops with Python s PyOS_InputHook API A new command line flag gui controls GUI support and it can also be enabled after Python startup via the new gui magic This requires some changes if you want to execute GUI using scripts inside Python see the GUI support section for more details A two process architecture The Qt console is the first use of a new model that splits Python between a kernel process where code is executed and a client that handles user interaction We plan on also providing terminal and web browser based clients using this infrastructure in future releases This model allows multiple clients to interact with an Python process through a well documented messaging protocol using the ZeroMQ networking library Refactoring the entire codebase has been refactored in order to make it more modular and easier to contribute to Python has traditionally been a hard project to participate because the old codebase 2 8 0 11 Series 153 IPython Documentation Release 1 2 1 An Afternoon Hack Aut was very monolithic We hope this ongoing restructuring will make it easier for new developers to join us Vim integration Vim can be configured to seamlessly control an Python kernel see the files in docs examples vim for the full details This work was done by Paul Ivanov who prepared a nice video demonstration of the feat
219. ble and this extends to the loopback of all connected clients which have opened a loopback port that is redirected to the controller s loopback port SSH Since ZeroMQ provides no security SSH tunnels are the primary source of secure connections A connec tor file such as ipcontroller client json will contain information for connecting to the controller possibly including the address of an ssh server through with the client is to tunnel The Client object then creates tunnels using either OpenSSH or Paramiko depending on the platform If users do not wish to use OpenSSH or Paramiko or the tunneling utilities are insufficient then they may construct the tunnels them selves and simply connect clients and engines as if the controller were on loopback on the connecting machine Authentication To protect users of shared machines HMAC digests are used to sign messages using a shared key The Session object that handles the message protocol uses a unique key to verify valid messages This can be any value specified by the user but the default behavior is a pseudo random 128 bit number as generated by uuid uuid4 This key is used to initialize an HMAC object which digests all messages and includes that digest as a signature and part of the message Every message that is unpacked on Controller Engine and Client will also be digested by the receiver ensuring that the sender s key is the same as the receiver s No messa
220. book 1028 Cleaner minimum version comparison 998 defer to stdlib for path get_home_dir 1033 update copyright to 2011 20xx 2011 1032 Intercept lt esc gt avoid closing websocket on Firefox 1030 use pyzmq tools where appropriate 1029 Restore pspersistence including store magic as an extension 1025 Dollar escape 999 Fix issue 880 more useful message to user when paste fails 938 changes to get ipython el to work with the latest python mode el 1012 Add logout button 1020 Dollar formatter for shell calls 1019 Use repr to make quoted strings 1008 don t use crash_handler by default 1003 Drop consecutive duplicates when refilling readline history 997 don t unregister interrupted post exec functions 996 add Integer traitlet 1016 Fix password hashing for Python 3 1014 escape minus signs in manpages 1013 NumPyExampleDocstring link was pointing to raw file 1011 Add hashed password support 1005 Quick fix for os system requiring str parameter e 994 Allow latex formulas in HTML output e 955 Websocket Adjustments e 979 use system_raw in terminal even on Windows e 989 fix arguments for commands in _process_posix e 991 Show traceback continuing to start kernel if pylab init fails e 981 Split likely multiline text when writing JSON notebooks e 957 allow change of png DPI in inline backend e 968 add wantDirectory to ipdoctest so that directories will be checked fo
221. books Clusters To import a notebook drag the file onto the listing below or click here a New Notebook home fperez ipython ipython docs examples notebooks 00_notebook_tour Shutdown 01_notebook_ introduction Delete 2 4 0 13 Series 73 IPython Documentation Release 1 2 1 An Afternoon Hack Cluster management The notebook dashboard can now also start and stop clusters thanks to a new tab in the dashboard user inter IPy IPython Dashboard amp TA 127 0 0 1 8888 tab2 Ox Oa IPiy Notebook Notebooks Clusters IPython parallel computing clusters profile status of engines action default stopped Start foo stopped SC Start iptest stopped ts Start mpi stopped Ot Start nbserver stopped CO Start face This interface allows for each profile you have configured to start and stop a cluster and optionally over ride the default number of engines corresponding to that configuration While this hides all error reporting once you have a configuration that you know works smoothly it is a very convenient interface for controlling your parallel resources New notebook format The notebooks saved now use version 3 of our format which supports heading levels as well as the concept of raw text cells that are not rendered as Markdown These will be useful with converters we are develop ing to pass raw markup say LaTeX That conversion code is still under heavy development and not quite r
222. ccess to that machine then you would connect to it with In 2 c Client path to my ipcontroller client json sshserver me myhub Jexample com Where myhub example com is the url or IP address of the machine on which the Hub process is running or another machine that has direct access to the Hub s ports The SSH server may already be specified in ipcontroller client json if the controller was instructed at its launch time You are now ready to learn more about the Direct and LoadBalanced interfaces to the controller 5 2 Starting the IPython controller and engines To use Python for parallel computing you need to start one instance of the controller and one or more instances of the engine The controller and each engine can run on different machines or on the same machine Because of this there are many different possibilities Broadly speaking there are two ways of going about starting a controller and engines e In an automated manner using the ipcluster command e Ina more manual way using the ipcont roller and ipengine commands 264 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack This document describes both of these methods We recommend that new users start with the ipcluster command as it simplifies many common usage cases 5 2 1 General considerations Before delving into the details about how you can start a controller and engines
223. ce Note Singletons are not strictly enforced you can have many instances of a given singleton class but the instance method will always return the same one Having described these main concepts we can now state the main idea in our configuration system con figuration allows the default values of class attributes to be controlled on a class by class basis Thus all instances of a given class are configured in the same way Furthermore if two instances need to be configured differently they need to be instances of two different classes While this model may seem a bit restrictive we have found that it expresses most things that need to be configured extremely well How ever it is possible to create two instances of the same class that have different trait values This is done by overriding the configuration Now we show what our configuration objects and files look like 6 1 2 Configuration objects and files A configuration file is simply a pure Python file that sets the attributes of a global pre created configuration object This configuration object is a Config instance While in a configuration file to get a reference to this object simply call the get_config function We inject this function into the global namespace that the configuration file is executed in Here is an example of a super simple configuration file that does nothing c get_config Once you get a reference to the configurat
224. cefully terminate any kernel that is still alive to avoid leaving stray processes in the user s machine Message type shutdown_request content restart bool whether the shutdown is final or precedes a restart Message type shutdown_reply content restart bool whether the shutdown is final or precedes a restart Note When the clients detect a dead kernel thanks to inactivity on the heartbeat socket they simply send a forceful process termination signal since a dead process is unlikely to respond in any useful way to messages 7 2 6 Messages on the PUB SUB socket Streams stdout stderr etc Message type st ream content The name of the stream is one of stdout stderr name str The data is an arbitrary string to be written to that stream data str Display Data This type of message is used to bring back data that should be diplayed text html svg etc in the frontends This data is published to all frontends Each message can have multiple representations of the data it is up 7 2 Messaging in IPython 397 IPython Documentation Release 1 2 1 An Afternoon Hack to the frontend to decide which to use and how A single message should contain all possible representations of the same information Each representation should be a JSON able data structure and should be a valid MIME type Some questions remain about th
225. class PromptManager which has traits for in_template in2_template the continuation prompt out_template and rewrite_template This uses Python s string formatting system so you can use time and cwd although we have preserved the abbreviations from previous versions e g prompt number and w working directory For the list of available fields refer to the source of TPython core prompts py e The class inheritance of the Launchers in IPython parallel apps launcher used by ip cluster has changed so that trait names are more consistent across batch systems This may require a few renames in your config files if you customized the command line args for launching controllers and engines The configurable names have also been changed to be clearer that they point to class names and can now be specified by name only rather than requiring the full import path of each class e g IPClusterEngines engine_launcher IPython parallel apps launcher MPIExecEngineSetLau IPClusterStart controller_launcher IPython parallel apps launcher SSHControllerLaun would now be specified as TPClusterEngines engine_launcher_class MPI TPClusterStart controller_launcher_class SSH The full path will still work and is necessary for using custom launchers not in Python s launcher module 2 6 0 12 Series 131 IPython Documentation Release 1 2 1 An Afternoon Hack Fur
226. code on Python 3 e 1077 allow the notebook to run without MathJax e 1072 If object has a getdoc method override its normal docstring e 1059 Switch to simple _ JPYTHON__ global e 1070 Execution count after SyntaxError e 1098 notebook config section UI e 1101 workaround spawnb missing from pexpect __all__ e 1097 typo should fix 1095 e 1099 gtconsole export xhtml utf8 e 1083 Prompts e 1081 Fix wildcard search for updated namespaces e 1084 write busy in notebook window title e 1078 PromptManager fixes e 1064 Win32 shlex e 1069 As you type completer fix on Firefox e 1039 Base of an as you type completer e 1065 Qtconsole fix racecondition 507 Prompt manager 1056 Warning in code qtconsole ssh X 1036 Clean up javascript based on js2 mode feedback 1052 Pylab fix 648 Usermod 969 Pexpect u 1007 Fix paste cpaste bug and refactor cleanup that code a lot 506 make ENTER on a previous input field replace current input buffer 1040 json jsonapi cleanup 1042 fix firefox windows break line on empty prompt number 1015 emacs freezes when tab is hit in ipython with latest python mode 1023 flush stdout stderr at the end of kernel init 138 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 956 Generate All magics menu live 1038 Notebook don t change cell when selecting code using shiftt tup down 987 Add Tooltip to note
227. com gt Google Summer of Code 2010 Qt notebook frontend support with ZeroMQ Paul Ivanov lt pivanov314 AT gmail com gt multiline specials improvements Dav Clark lt davclark AT berkeley edu gt traitlets improvements David Warde Farley lt wardefar AT iro umontreal ca gt bugfixes to timeit input autoindent manage ment and Qt console tooltips Darren Dale lt dsdale24 AT gmail com gt traits based configuration system Qt support Jose Unpingco lt unpingco gmail com gt authored multiple tutorials and screencasts teaching the use of Python both for interactive and parallel work available in the documentation part of our website Dan Milstein lt danmil AT comcast net gt A bold refactor of the core prefilter machinery in the Python interpreter Jack Moffit lt jack AT xiph org gt Bug fixes including the infamous color problem This bug alone caused many lost hours and frustration many thanks to him for the fix I ve always been a fan of Ogg amp friends now I have one more reason to like these folks Jack is also contributing with Debian packaging and many other things Alexander Schmolck lt a schmolck AT gmx net gt Emacs work bug reports bug fixes ideas lots more The ipython el mode for X Emacs is Alex s code providing full support for Python under X Emacs Andrea Riciputi lt andrea riciputi AT libero it gt Mac OSX information Fink package management Gary Bishop lt gb AT cs unc edu gt Bug repo
228. completion for zsh move DEFAULT_STATIC_FILES_PATH to Python html minor bower tweaks Default color output for ls on OSX fix dot syntax error in inheritance diagram raise UnsupportedOperation on iostream fileno Notebook support for a reverse proxy which handles SSL make qtconsole size at startup configurable adding stream kwarg to current new_output IPEP 10 kernel side filtering of display formats add redirect handler for notebooks by name support non modules in require Stateful line transformers fix some O N and O N 2 operations in parallel map forward stdout from forked processes use Python 3 style for pretty printed sets closes 3045 3123 for tornado lt version 3 0 minor heading link tweaks Strip useless ANSI escape codes in notebook Prevent errors when pressing arrow keys in an empty notebook quick dev installation instructions Push pandas dataframes to R magic Don t monkeypatch doctest during Python startup fix argparse version check set spellcheck false in CodeCell inputarea add anchors to heading cells 36 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 3117 PR 3097 PR 3118 PR 3115 PR 2640 PR 3070 PR 3082 PR 3078 PR 3054 PR 3073 PR 3083 PR 3080 PR 3079 PR 3076 PR 3063 PR 3075 PR 3051 PR 2999 PR 2997 PR 3033 PR 3062 PR 3046 PR 3038 PR 3032 PR 3022 PR 3024 PR 3013
229. completion history problems to be bugs for Python if you are using libedit To get a working readline module just do with setuptool1s installed S easy_install readline Note Other Python distributions on OS X such as fink MacPorts and the official python org binaries already have readline installed so you likely don t have to do this step When Python is installed with setuptools e g using the easy_install command readline is added as a dependency on OS X and PyReadline on Windows and will be installed on your system How ever if you do not use setuptools you may have to install one of these packages yourself 3 4 2 nose To run the Python test suite you will need the nose package Nose provides a great way of sniffing out and running all of the Python tests The simplest way of getting nose is to use easy_install1 or pip S pip install nose Another way of getting this is to do S pip install ipython test For more installation options see the nose website Once you have nose installed you can run Python s test suite using the iptest command 198 Chapter 3 Installation IPython Documentation Release 1 2 1 An Afternoon Hack S iptest 3 4 3 pexpect The pexpect package is used in IPython s irunner script as well as for managing subprocesses Python now includes a version of pexpect in Python external but if you have installed pexpect
230. configured to use the mpiexec or mpirun commands to start MPI processes If these are satisfied you can create a new profile 5 2 Starting the IPython controller and engines 267 ngineSetLau IPython Documentation Release 1 2 1 An Afternoon Hack ipython profile create parallel profile mpi and edit the file ITIPYTHONDIR profile_mpi ipcluster_config py There instruct ipcluster to use the MPI launchers by adding the lines c IPClusterEngines engine_launcher_class MPIEngineSetLauncher If the default MPI configuration is correct then you can now start your cluster with ipcluster start n 4 profile mpi This does the following 1 Starts the Python controller on current host 2 Uses mpiexec to start 4 engines If you have a reason to also start the Controller with mpi you can specify c IPClusterStart controller_launcher_class MPIControllerLauncher Note The Controller will not be in the same MPI universe as the engines so there is not much reason to do this unless sysadmins demand it On newer MPI implementations such as OpenMPD this will work even if you don t make any calls to MPI or call MPI_Init However older MPI implementations actually require each process to call MPI_Init upon starting The easiest way of having this done is to install the mpi4py mpi4py package and then specify the c MPI use option in ipengine_config py
231. ct Execution payloads The notion of an execution payload is different from a return value of a given set of code which normally is just displayed on the pyout stream through the PUB socket The idea of a payload is to allow special types of code typically magics to populate a data container in the Python kernel that will be shipped back to the caller via this channel The kernel has an API for this in the PayloadManager ip payload_manager write_payload payload_dict which appends a dictionary to the list of payloads The payload API is not yet stabilized and should probably not be supported by non Python kernels at this time In such cases the payload list should always be empty When status is error the following extra fields are present ename str Exception name as a string Tevalue str Exception value as a string The traceback will contain a list of frames represented each as a string For now we ll stick to the existing design of ultraTB which controls exception level of detail statefully But eventually we ll want to grow into a model where more information is collected and packed into the traceback object with clients deciding how little or how much of it to unpack But for now let s start with a simple list of strings since that requires only minimal changes to ultratb as written traceback list o SHS SHS Gis SHS SHS SH As When status is abor
232. ct of the GIL in our previous code was something we could simply not work around given that Twisted is itself a Python library So while Twisted is a very capable framework we think ZeroMQ fits our needs much better and we hope you will find the change to be a significant improvement in the long run Our manual contains a full description of how to use IPython for parallel computing and the tutorial pre sented by Min Ragan Kelley at the SciPy 2011 conference provides a hands on complement to the reference docs Refactoring As of this release a signifiant portion of Python has been refactored This refactoring is founded on a number of new abstractions The main new classes that implement these abstractions are e IPython utils traitlets HasTraits e IPython config configurable Configurable e IPython config application Application e IPython config loader ConfigLoader e IPython config loader Config We are still in the process of writing developer focused documentation about these classes but for now our configuration documentation contains a high level overview of the concepts that these classes express The biggest user visible change is likely the move to using the config system to determine the command line arguments for Python applications The benefit of this is that all configurable values in Python are exposed on the command line but the syntax for specifying values has changed The gist is that assign
233. ct that the decorators use Graph Dependencies Sometimes you want to restrict the time and or location to run a given task as a function of the time and or location of other tasks This is implemented via a subclass of set called a Dependency A Depen dency is just a set of msg_ids corresponding to tasks and a few attributes to guide how to decide when the Dependency has been met The switches we provide for interpreting whether a given dependency set has been met anylall Whether the dependency is considered met if any of the dependencies are done or only after all of them have finished This is set by a Dependency s a11 boolean attribute which defaults to True success default True Whether to consider tasks that succeeded as fulfilling dependencies 298 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack failure default False Whether to consider tasks that failed as fulfilling dependencies using fail ure True success False is useful for setting up cleanup tasks to be run only when tasks have failed Sometimes you want to run a task after another but only if that task succeeded In this case success should be True and failure should be False However sometimes you may not care whether the task succeeds and always want the second task to run in which case you should use success failure True The default behavior is to only use successes There are other
234. d PR 2068 record sysinfo in sdist PR 2067 update tools release_windows py PR 2065 Fix parentheses typo PR 2062 Remove duplicates and auto generated files from repo PR 2061 use explicit tuple in exception e PR 2060 change minus to or hy in manpages 3940 3946 3957 3894 3887 3703 3931 3936 3941 3926 3932 3928 3774 3916 3915 3895 3912 3636 3897 Install process documentation overhaul The PDF option for post should work with lowercase Notebook help page broken in Firefox nbconvert test failure 1 0 0a1 shows blank screen in both firefox and chrome windows 7 nbconvert Output options names and documentataion Tab completion not working during debugging in the notebook Ipcluster plugin is not working with Ipython 1 0dev Python Notebook kernel crash on Win7x64 Ending Notebook renaming dialog with return creates new line Incorrect empty docstring Passing variables to script from the workspace Notebooks with spaces in their names breaks nbconvert latex graphics tornado needs its own check Link to Parallel examples found on GitHub broken in docs Keyboard shortcuts box in notebook doesn t fit the screen IPython utils fails automated test for RC1 1 0 0 Code cell missing highlight on load under Windows ipython3 nbconvert C blabla first_try ipynb to latex post PDF POST processing action fails because of a bad parameter 3
235. d win Now the QApplication itself holds a reference to win so it will never be garbage collected until the appli cation itself is destroyed 4 5 8 Regressions There are some features where the qt console lags behind the Terminal frontend e cmd input Due to our use of pexpect we cannot pass input to subprocesses launched using the P escape so you should never call a command that requires interactive input For such cases use the terminal Python This will not be fixed as abandoning pexpect would significantly degrade the console experience 4 6 The IPython Notebook 4 6 1 Introduction The notebook extends the console based approach to interactive computing in a qualitatively new direction providing a web based application suitable for capturing the whole computation process developing doc umenting and executing code as well as communicating the results The Python notebook combines two components A web application a browser based tool for interactive authoring of documents which combine explanatory text mathematics computations and their rich media output Notebook documents a representation of all content visible in the web application including inputs and outputs of the computations explanatory text mathematics images and rich media representations of ob jects See also 244 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack S
236. d as line magic return line else print Called as cell magic return line cell We delete these to avoid name conflicts for automagic to work del lmagic lcmagic You can also create magics of all three kinds by inheriting from the Python core magic Magics class This lets you create magics that can potentially hold state in between calls and that have full access to the main Python object This code can be put in any Python module it does not require IPython itself to be running already It only creates the magics subclass but doesn t instantiate it yet from IPython core magic import Magics magics_class line_magic cell_magic line_cell_magic The class MUST call this class decorator at creation time magics_class class MyMagics Magics line_magic def lmagic self line my line magic print Full access to the main IPython object self shell print Variables in the user namespace self shell user_ns keys return line cell_ magic def cmagic self line cell my cell magic return line cell line_cell_magic def lcmagic self line cell None Magic that works both as lcomagic and as lcmagic if cell is None print Called as line magic 4 3 IPython reference 211 IPython Documentation Release 1 2 1 An Afternoon Hack return line else print Called as cell magic return line cell In order to actually use
237. d by other projects Public APIs For the first time since 0 10 sorry everyone there is an official public API for starting Python from IPython import start_ipython start_ipython This is what packages should use that start their own IPython session but don t actually want embedded Python most cases IPython embed is used for embedding Python into the calling namespace similar to calling Pdb set_trace whereas start_ipython will start a plain Python session loading config and startup files as normal We also have added from IPython import get_ipython Which is a library function for getting the current Python instance and will return None if no Python instance is running This is the official way to check whether your code is called from inside an Python session If you want to check for Python without unnecessarily importing Python use this function def get_ipython return IPython instance if there is one None otherwis import sys if IPython in sys modules import IPython return IPython get_ipython CM a Core e The input transformation framework has been reworked This fixes some corner cases and adds more flexibility for projects which use IPython like SymPy amp SAGE For more details see Custom input transformation 8 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Exception types can
238. ding when the cell is executed 4 6 5 Basic workflow The normal workflow in a notebook is then quite similar to a standard Python session with the difference that you can edit cells in place multiple times until you obtain the desired results rather than having to rerun separate scripts with the run magic command Typically you will work on a computational problem in pieces organizing related ideas into cells and mov ing forward once previous parts work correctly This is much more convenient for interactive exploration than breaking up a computation into scripts that must be executed together as was previously necessary especially if parts of them take a long time to run 248 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack At certain moments it may be necessary to interrupt a calculation which is taking too long to complete This may be done with the Kernel Interrupt menu option or the Ctrl m i keyboard shortcut Similarly it may be necessary or desirable to restart the whole computational process with the Kernel Restart menu option or Ctrl m shortcut A notebook may be downloaded in either a ipynb or py file from the menu option File Download as Choosing the py option downloads a Python py script in which all rich output has been removed and the content of markdown cells have been inserted as comments See also Running Code in the Python Notebook ex
239. diting the dochelp core completer Clean up excessive and unused code docs looks like a file path might have been accidentally pasted in the middle of a word Fix link to Chris Fonnesbeck blog post about 0 11 highlights e 1943 add screenshot and link into releasenotes e 1570 notebook remove left panel references from example 102 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 1954 update some example notebooks 2048 move _encode_binary to jsonutil encode_images 2050 only add quotes around xunit file on Windows 2047 disable auto scroll on mozilla 1258 Magic paste error 2015 Fixes for paste with special transformations 760 Windows test runner fails if repo path contains spaces 2046 Iptest unicode 1939 Namespaces 2042 increase auto scroll threshold to 100 lines 2043 move RemoteError import to top level 641 In magic help remove duplicate aliases 2036 Yalias_magic 1968 Proposal of icons for ipynb files 825 keyboardinterrupt crashes gtk gui when gtk set_interactive is not available 1971 Remove duplicate magics docs 2040 Namespaces for cleaner public APIs 2039 ipython parallel import exception 2035 Getdefaultencoding test error with sympy 0 7 1_git 2037 remove ipython qtconsole gui script 1516 ipython qtconsole script isn t installed for Python 2 x 1297 ipython p sh is in documentation but doesn t work
240. do in the tests 148 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 960 934 e 870 943 e 947 903 908 935 869 904 e 773 e 915 886 e 636 808 e 927 e 919 e 181 e 917 e 818 907 867 e 211 e 900 e 513 e 906 638 e 710 436 e 207 692 690 crash on syntax error on Windows XP The latest ipython branch doesn t work in Chrome zmq version detection HISTIGNORE for Python qtconsole segfaults at startup Expose a magic to control config of the inline pylab backend bad user config shouldn t crash Python Typing break causes Python to crash Tab completion of shows no output post 0 10 x whos under pypy1 6 check_security_dir and check_pid_dir fail on network filesystem OS X Lion Terminal app line wrap problem Notebook kernel crash when specifying notebook dir on commandline debugger py pydb broken Ctrl C during reset confirm message crash Qtconsole Using return outside a function crashes ipython Pop up segfault when moving cursor out of qtconsole window cls command does not work on windows documentation typos run does not work with non ascii characeters in path Errors in custom completer functions can crash Python doc notebook password authentication howto paste command not working Tab key should insert 4 spaces in qt console Qt console cannot insert new lines
241. dy using Perl e ol prun run cell body with profiler cell extension of Sprun e ol o9 python3 run cell body using Python 3 o sruby run cell body using Ruby e ol script run cell body with the script specified in the first line e ol sh run cell body using sh o sx run cell with system shell and capture process output cell extension of sx 2 4 0 13 Series 77 IPython Documentation Release 1 2 1 An Afternoon Hack e Ssystem run cell with system shell is an alias to this e Stimeit time the execution of the cell extension of St imeit This is what some of the script related magics look like in action IPython also creates aliases for a few common interpreters such as bash ruby perl etc These are all equivalent to script lt name gt In 4 ruby puts Hello from Ruby RUBY_VERSION Hello from Ruby 1 8 7 In 5 bash echo hello from BASH hello from usr local bin bash In addition we have also a number of extensions that provide specialized magics These typically require additional software to run and must be manually loaded via 3 load_ext lt extension name gt but are extremely useful The following extensions are provided Cython magics extension cythonmagic This extension provides magics to automatically build and com pile Python extension modules using the Cython language You must install Cython separately as well as
242. e D in DAG stands for Directed This means that each edge has a direction associated with it So we can interpret the edge a b as meaning that b depends on a whereas the edge b a would mean a depends on b The A is Acyclic meaning that there must not be any closed loops in the graph This is important for dependencies because if a loop were closed then a task could ultimately depend on itself and never be able to run If your workflow can be described as a DAG then it is impossible for your dependencies to cause a deadlock 5 12 2 A Sample DAG Here we have a very simple 5 node DAG With NetworkX an arrow is just a fattened bit on the edge Here we can see that task 0 depends on nothing and can run immediately 1 and 2 depend on 0 3 depends on 1 and 2 and 4 depends only on 1 A possible sequence of events for this workflow 0 Task 0 can run right away 1 0 finishes so 1 2 can start 2 1 finishes 3 is still waiting on 2 but 4 can start right away 3 2 finishes and 3 can finally start Further taking failures into account assuming all dependencies are run with the default suc cess True failure False the following cases would occur for each node s failure O fails all other tasks fail as Impossible 1 2 can still succeed but 3 4 are unreachable 2 3 becomes unreachable but 4 is unaffected 3 and 4 are terminal and can have no effect on other nodes The code to generate the simple DAG
243. e This makes debugging plotting etc in parallel scenarios vastly easier Parallel engines can always execute arbitrary Python code that is code that has magics shell extensions etc In combination with the 3 px magics it is thus extremely natural for example to send to all engines a block of Cython or R code to be executed via the new Cython and R magics For example this snippet would send the R block to all active engines in a cluster ole oe px R ole oe R code goes here 2 4 0 13 Series 85 IPython Documentation Release 1 2 1 An Afternoon Hack e It is possible to embed not only an interactive shell with the TPython embed call as always but now you can also embed a kernel with IPython embed_kernel Embedding an Python ker nel in an application is useful when you want to use TPython embed but don t have a terminal attached on stdin and stdout e The new IPython parallel bind_kernel allows you to promote Engines to listening Kernels and connect QtConsoles to an Engine and debug it directly In addition having a single core object through our entire architecture also makes the project conceptually cleaner easier to maintain and more robust This took a lot of work to get in place but we are thrilled to have this major piece of architecture finally where we d always wanted it to be Official Public API We have begun organizing our API for easier public use with an eye
244. e This make the remote namespaces of the engines appear as a local dictionary Underneath these methods call apply In 51 dview a foo bar In 52 dview a Out 52 amp oo bar i fo00 bar too bar foo bar Scatter and gather Sometimes it is useful to partition a sequence and push the partitions to different engines In MPI lan guage this is know as scatter gather and we follow that terminology However it is important to remember that in IPython s Client class scatter is from the interactive Python session to the engines and gather is from the engines back to the interactive Python session For scatter gather operations be tween engines MPI pyzmgq or some other direct interconnect should be used In 58 dview scatter a range 16 Out 58 None None None None In 59 dview a Out 59 0r Ly 22 als Mye Biz Op Tir 8 97 L0 HL T127 13 24 15 In 60 dview gather a Out 60 Op dy Zee 3p Ar Op OR Ty Se 9 10 14 12 23 24 15 5 3 6 Other things to look at How to do parallel list comprehensions In many cases list comprehensions are nicer than using the map function While we don t have fully parallel list comprehensions it is simple to get the basic effect using scatter and gather In 66 dview scatter x range 64 In 67 Spx y i x 10 for i in x Parallel e
245. e in seconds for the scheduler to wait for dependencies to be met before failing with a DependencyTime out execute and run For executing strings of Python code Direct View s also provide an execute anda run method which rather than take functions and arguments take simple strings execute simply takes a string of Python code to execute and sends it to the Engine s run is the same as execute but for a file rather than a string It is simply a wrapper that does something very similar to execute open f read Note TODO Examples for execute and run 5 13 3 Views The principal extension of the Client is the View class The client is typically a singleton for connecting to a cluster and presents a low level interface to the Hub and Engines Most real usage will involve creating one or more View objects for working with engines in various ways DirectView The Direct View is the class for the Python Multiplexing Interface Creating a DirectView DirectViews can be created in two ways by index access to a client or by a clients view method Index access to a Client works in a few ways First you can create DirectViews to single engines simply by accessing the client by engine id In 2 tero Out 2 lt DirectView 0 gt You can also create a DirectView with a list of engines 5 13 Details of Parallel Computing with IPython 335 IPython Documentation Release 1 2 1 An Afternoon
246. e 1 2 1 An Afternoon Hack In detail See the git page for the most recent information Have a look at the github install help pages available from github help There are good instructions here http book git scm com 2_installing_git html 7 1 2 Following the latest source These are the instructions if you just want to follow the latest ipython source but you don t need to do any development for now The steps are e Install git e get local copy of the git repository from github e update local copy from time to time Get the local copy of the code From the command line git clone git github com ipython ipython git You now have a copy of the code tree in the new ipython directory Updating the code From time to time you may want to pull down the latest code Do this with cd ipython git pull The tree in ipython will now have the latest changes from the initial repository 7 1 3 Making a patch You ve discovered a bug or something else you want to change in ipython excellent You ve worked out a way to fix it even better You want to tell us about it best of all The easiest way is to make a patch or set of patches Here we explain how Making a patch is the simplest and quickest but if you re going to be doing anything more than simple quick things please consider following the Git for development model instead 372 Chapter 7 IPython developer s guide IPython Docum
247. e at gmail com gt Bernard Paulus lt bprecyclebin at gmail com gt Bradley M Froehle lt brad froehle at gmail com gt Brian E Granger lt ellisonbg at gmail com gt Christian Boos lt cboos at bct technology com gt Daniel Velkov lt danielv at mylife com gt Erik Tollerud lt erik tollerud at gmail com gt Evan Patterson lt epatters at enthought com gt Felix Werner lt Felix Werner at kit edu gt Fernando Perez lt Fernando Perez at berkeley edu gt Gabriel lt g2p code at gmail com gt Grahame Bowland lt grahame at angrygoats net gt Hannes Schulz lt schulz at ais uni bonn de gt Jens Hedegaard Nielsen lt jenshnielsen at gmail com gt Jonathan March lt jmarch at enthought com gt 132 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack J rgen Stenarson lt jorgen stenarson at bostream nu gt Julian Taylor lt jtaylor debian at googlemail com gt Kefu Chai lt tchaikov at gmail com gt macgyver lt neil rabinowitz at merton ox ac uk gt Matt Cottingham lt matt cottingham at gmail com gt Matthew Brett lt matthew brett at gmail com gt Matthias BUSSONNIER lt bussonniermatthias at gmail com gt Michael Droettboom lt mdboom at gmail com gt Nicolas Rougier lt Nicolas Rougier at inria fr gt Olivier Verdier lt olivier verdier at gmail com gt Omar Andres Zapata Mesa lt andresete chaos at gmail com gt Pablo Winant lt pablo winant at gmail com
248. e because it is self signed If you want to have a fully compliant certificate that will not raise warnings it is possible but rather involved to obtain one as explained in detail in this tutorial Keep in mind that when you enable SSL support you will need to access the notebook server over https not over plain http The startup message from the server prints this but it is easy to overlook and think the server is for some reason non responsive 4 8 2 Running a public notebook server If you want to access your notebook server remotely via a web browser you can do the following Start by creating a certificate file and a hashed password as explained above Then create a custom profile for the notebook with the following command line type ipython profile create nbserver In the profile directory just created edit the file ipython_notebook_config py By default the file has all fields commented the minimum set you need to uncomment and edit is the following c get_config Kernel config c IPKernelApp pylab inline if you want plotting support always Notebook config c NotebookApp certfile u absolute path to your certificate mycert pem c NotebookApp ip x c NotebookApp open_browser False c NotebookApp password u shal bcd259ccf your hashed password here It is a good idea to put it on a known fixed port c NotebookApp port 9999 You can then start
249. e cluster Associated with each cluster profile is a cluster directory This cluster directory is a specially named direc tory typically located in the ipython subdirectory of your home directory that contains the configura tion files for a particular cluster profile as well as log files and security keys The naming convention for cluster directories is profile_ lt profile name gt Thus the cluster directory for a profile named foo would be ipython cluster_foo To create a new cluster profile named mycluster and the associated cluster directory type the following command at the Windows Command Prompt ipython profile create parallel profile mycluster The output of this command is shown in the screenshot below Notice how ipcluster prints out the location of the newly created profile directory Z gt ipython profile create mycluster parallel ProfileCreate Generating default config file u blue domainusers bgranger ipyth ProfileCreate Generating default config file u blue domainusers bgranger ipyth ProfileCreate Generating default config file u blue domainusers bgranger ipyth ProfileCreate Generating default config file u blue domainusers bgranger ipyth ProfileCreate Generating default config file u blue domainusers bgranger ipyth Z gt 318 Chapter 5 Using IPython for parallel computing IPython
250. e eae Bee hoe ow Be dae Panes 5 8 Python s Task Database c 24 0 24408 fae 4 Sa SG Oe Ba RoR Sa Se PH Se Sew HS 5 9 Security details of Pythons gg ak a OA Ae OS Se ee Aa a a BS 5 10 Getting started with Windows HPC Server 2008 2 0 0000 5 11 Parallel examples socso 4 hk eae a ee a ee ew we 512 DAG Dependencies jio sls ask ke wlan Sie aoe Sane ee So te ah Goh a Balle a 5 13 Details of Parallel Computing with Python 000 5 14 Transitioning from Python kernel to IPython parallel 6 Configuration and customization 6 1 Overview of the Python configuration system 2 02 002 eee eee ee 6 2 WPyihOmeXtensions e e 68 asa G Meh 444 oo oo ewe RS ee Gee ai aie 6 3 Configuring the ipython command line application 2 04 6 4 Integrating your objects with IPython 2 20 a 6 5 Editor configuration o coc eoe 5 82 8 e eie cd baudbavetwae dws Bbw bd 6 6 Custom input transformation c sae e es da ea ee eaa eee 6 7 Outdated configuration information that might still be useful noana 7 IPython developer s guide 7 1 Working with IPython source code e ade saci a ciowa aee ee 17 2 M ss ging miIPython s i s x iaa mea a aei aae ee eae ee wee ee we eS 7 3 Messaging for Parallel Computing s e s e s eeoa aia ee 7 4 Connection Diagrams of The IPython ZMQ Cluster anaoa 8 The IPython API 9 About IPython OF Credits saer e T gg dad ed amp
251. e fperez ipython In 2 cd 1 now cd is just a variable In L314 sed wx and doesn t work as a function anymore File lt ipython input 3 9fedb3aff56c gt line 1 cd SyntaxError invalid syntax In 4 cd but cd always works home fperez In 5 del cd if you remove the cd variable automagic works again In 6 cd ipython SS home fperez ipython Defining your own magics There are two main ways to define your own magic functions from standalone functions and by inheriting from a base class provided by Python IPython core magic Magics Below we show code you can place in a file that you load from your configuration such as any file in the startup subdirectory of your default Python profile First let us see the simplest case The following shows how to create a line magic a cell one and one that works in both modes using just plain functions from IPython core magic import register_line_magic register_cell_magic register_line_cell_magic register_line_ magic 210 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack def lmagic line my line magic return line register_cell_ magic def cmagic line cell my cell magic return line cell register_line_cell_ magic def lcmagic line cell None Magic that works both as lcomagic and as lcmagic if cell is None print Calle
252. e import management for iptest py 31 Issues in magic_whos code 52 Document testing process better 44 Merge history from multiple sessions 182 ipython q4thread in version 10 1 not starting properly 143 Ipython gui wx ipython_view IPShellWidget ignores user _ns arguments 127 edit does not work on filenames consisted of pure numbers 126 Can t transfer command line argument to script 28 Offer finer control for initialization of input streams 58 ipython change char Oxe9 to 4 spaces 68 Problems with Control C stopping ipcluster on Windows Python2 6 24 ipcluster does not start all the engines 240 Incorrect method displayed in psource 120 inspect getsource fails for functions defined on command line 212 Python ignores exceptions in the first evaulation of class attrs 108 ipython disables python logger 100 Overzealous introspection 18 Yocpaste freeze sync frontend 200 Unicode error when starting ipython in a folder with non ascii path 130 Deadlock when importing a module that creates an Python client 134 multline block scrolling 46 Input to timeit is not preparsed 285 ipcluster local n 4 fails 205 In the Qt console Tab should insert 4 spaces when not completing 145 Bug on MSW sytems idle can not be set as default Python editor Fix Suggested 77 ipython oops in cygwin 121 If plot windows are closed via window controls no more plotting is possible 2 9 Issue
253. e in use The request will be sent to the frontend that made the request and an exception is raised if that frontend does not support stdin requests e g the notebook 673 e zmq version detection no longer uses simple lexicographical comparison to check minimum version which prevents 0 11 from working with pyzmq 2 1 10 PR 758 e A bug in PySide lt 1 0 7 caused crashes on OSX when tooltips were shown 711 these tooltips are now disabled on old PySide PR 963 e Python no longer crashes when started on recent versions of Python 3 in Windows 737 e Instances of classes defined interactively can now be pickled 29 PR 648 Note that pickling saves a reference to the class definition so unpickling the instances will only work where the class has been defined Backwards incompatible changes e Python connection information is no longer specified via ip port directly rather via json connection files These files are stored in the security directory and enable us to turn on HMAC message authen tication by default significantly improving the security of kernels Various utility functions have been added to IPython 1ib kernel for easier connecting to existing kernels e KernelManager now has one ip and several port traits rather than several ip port pair __addr traits This better matches the rest of the code where the ip cannot not be set separately for each channel e Custom prompts are now configured using a new
254. e is not neces sarily any Qt code running in the kernel and under most normal circumstances there isn t If however you specify matplotlib qt at the command line then there will be a QCoreApplication instance running in the kernel process along with user code To get a reference to this application do from PyOt4 import QtCore app OtCore QCoreApplication instance app will be None if there is no such instance A common problem listed in the PyQt4 Gotchas is the fact that Python s garbage collection will destroy Qt objects Windows etc once there is no longer a Python reference to them so you have to hold on to them For instance in 4 5 A Qt Console for Python 243 IPython Documentation Release 1 2 1 An Afternoon Hack def make_window win QtGui QMainWindow def make_and_return_window win QtGui QMainWindow return win make_window will never draw a window because garbage collection will destroy it before it is drawn whereas make_and_return_window lets the caller decide when the window object should be de stroyed If as a developer you know that you always want your objects to last as long as the process you can attach them to the QApplication instance itself do this just once app QtCore QCoreApplication instance app references set then when you create Windows add them to the set def make_window win QtGui QMainWindow app references ad
255. eShell s superclass Revert 1831 the __file__ injection in safe_execfile safe_execfile_ipy Autochange highlight with cell magics Add image message handler in ZMQTerminalInteractiveShell skip find_cmd when setting up script magics Catch sqlite DatabaseErrors in more places when reading the history database Don t catch ImportError when trying to unpack module functions enable IPC transport for kernels don t let log cleanup prevent engine start sys maxsize is the maximum length of a container allow iptest to be interrupted fix message built for engine dying during task Block until kernel termination after sending a kill signal Py3k Octal 0777 gt 00777 Detachable pager in notebook Fix installation of man pages in Python 3 add Python version to message headers Fix Issue 2366 clarify TaskScheduler hwm doc IndentationError display Add scroll_to_cell cell_number to the notebook unmock read the docs modules always perform requested trait assignments New option n to limit history search hits Adapt inline backend to changes in matplotlib 44 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 2392 PR 2387 PR 2370 PR 1715 PR 2378 PR 2373 PR 2368 PR 2327 PR 2362 PR 2346 PR 2352 PR 2358 PR 2365 PR 2364 PR 2354 PR 2339 PR 2345 PR 2347 PR 2305 PR 2338 PR 2192 PR 2329 PR 2337 PR 2331 PR 2317 PR
256. eady for prime time but we welcome help on this front so that we can merge it for full production use as soon as possible Note v3 notebooks can not be read by older versions of I Python but we provide a simple script that you can use in case you need to export a v3 notebook to share with a v2 user JavaScript refactoring All the client side JavaScript has been decoupled to ease reuse of parts of the machinery without having to build a full blown notebook This will make it much easier to communicate with an Python kernel from existing web pages and to integrate single cells into other sites without loading the full notebook document like UI PR 1711 This refactoring also enables the possibility of writing dynamic javascript widgets that are returned from Python code and that present an interactive view to the user with callbacks in Javascript executing calls to the Kernel This will enable many interactive elements to be added by users in notebooks 74 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack An example of this capability has been provided as a proof of concept in examples widgets that lets you directly communicate with one or more parallel engines acting as a mini console for parallel debugging and introspection Improved tooltips The object tooltips have gained some new functionality By pressing tab several times you can expand them to see more of a docstring kee
257. eant to work as a drop in replacement for the standard interactive interpreter As such any code which is valid python should execute normally under Python cases where this is not true should be reported as bugs It does however offer many features which are not available at a standard python prompt What follows is a list of these Caution for Windows users Windows unfortunately uses the V character as a path separator This is a terrible choice because V also represents the escape character in most modern programming languages including Python For this reason using character is recommended if you have problems with However in Windows commands flags options so you can not use it for the root directory This means that paths beginning at the root must be typed in a contrived manner like scopy opt foo bar txt tmp Magic command system Python will treat any line whose first character is a as a special call to a magic function These allow you to control the behavior of Python itself plus a lot of system type features They are all prefixed with a character but parameters are given without parentheses or quotes Lines that begin with signal a cell magic they take as arguments not only the rest of the current line but all lines below them as well in the current execution block Cell magics can in fact make arbitrary modifications to the input they receive which need not even be valid Python code
258. ease 1 2 1 An Afternoon Hack System command aliases The alias magic function allows you to define magic functions which are in fact system shell commands These aliases can have parameters alias alias_name cmd defines alias_name as an alias for cmd Then typing alias_name params will execute the system command cmd params from your under lying operating system You can also define aliases with parameters using s specifiers one per parameter The following example defines the parts function as an alias to the command echo first s second s where each s will be replaced by a positional parameter to the call to parts In 1 Salias parts echo first s second s In 2 parts A B first A second B In 3 parts A ERROR Alias lt parts gt requires 2 arguments 1 given If called with no parameters alias prints the table of currently defined aliases The rehashx magic allows you to load your entire PATH as ipython aliases See its docstring for further details Recursive reload The IPython 1lib deepreload module allows you to recursively reload a module changes made to any of its dependencies will be reloaded without having to exit To start using it do from IPython 1lib deepreload import reload as dreload Verbose and colored exception traceback printouts Python provides the option to see very detailed exception tracebacks which can be especially useful when
259. easing counter of all execution requests that are made with store_history True This counter is used to populate the In n Out n and _n variables so clients will likely want to display it in some form to the user which will typically but not necessarily be done in the prompts The value of this counter will be returned as the execution_count field of all execute_reply messages Execution results Message type execute_reply content One of ok OR error OR abort Status Str The global kernel counter that increases by one with each request that stores history This will typically be used by clients to display prompt numbers to the user If the request did not store history this wil be the current value of the counter in the kernel 390 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack execution_count int When status is ok the following extra fields are present payload will be a list of payload dicts Each execution payload is a dict with string keys that may have been produced by the code being executed It is retrieved by the kernel at the end of the execution and sent back to the front end which can take action on it as needed See main text for further details payload list dict Results for the user_variables and user_expressions user_variables dict user_expressions di
260. ecials True c AliasManager user_aliases Ukat FLS a 6 4 Integrating your objects with IPython 6 4 1 Tab completion To change the attributes displayed by tab completing your object define a__dir___ self method for it For more details see the documentation of the built in dir function 360 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack 6 4 2 Rich display The notebook and the Qt console can display richer representations of objects To use this you can de fine any of a number of _repr_ _ methods Note that these are surrounded by single not double underscores Both the notebook and the Qt console can display svg png and jpeg representations The notebook can also display html javascript and latex If the methods don t exist or return None it falls back to a standard repr For example class Shout object def init__ self text self text text def _repr_html_ self return lt h1 gt self text lt hi gt 6 4 3 Custom exception tracebacks Rarely you might want to display a different traceback with an exception IPython s own par allel computing framework does this to display errors from the engines To do this define a _render_traceback_ self method which returns a list of strings each containing one line of the traceback Please be conservative in using this feature by replacing the d
261. ect box 645 Standalone WX GUI support is broken 1296 1254 Wx gui example fixes the broken example for gui wx typo in notebooklist js breaks links 781 Users should be able to clone a notebook 1372 1374 1382 1370 1368 1379 1376 1358 1367 1359 1337 1339 1344 1348 1361 1364 1362 ipcontroller cleans up connection files unless reuse True remove calls to meaningless ZMQStream on_err Update RO for Notebook allow draft76 websockets Safari Ensure handler patterns are str not unicode Sage link on website homepage broken FWIW does not work with Chrome 16 0 912 77 Ubuntu 10 10 Cannot install ipython on Windows 7 64 bit Ctrl m t does not toggle output in chrome sympyprinting MathJax can t render root m n Tab in the notebook after should not indent only give a tooltip Notebook printing broken Ctrl M L does not toggle line numbering in htmlnotebook Ctrl M M does not switch to markdown cell Notebook bug fix branch avoid jsonlib returning Decimal Don t log complete contents of history replies even in debug 888 ReST support in notebooks 1205 1351 1243 1347 1355 1354 1346 1336 1331 notebook stores HTML escaped text in the file add IPython embed_kernel magic commands without are not completed properly in htmInotebook fix weird magic completion in notebook notebook html extends layout html now min and max in the
262. ect by setting hwm to a positive integer 1 being maximum load balancing a task will never be waiting if there is an idle engine and any larger number being a compromise between load balancing and latency hiding In practice some users have been confused by having this optimization on by default so the default value has been changed to 1 in Python 0 13 This can be slower but has more obvious behavior and won t result in assigning too many tasks to some engines in heterogeneous cases Pure ZMQ Scheduler For maximum throughput the pure scheme is not Python at all but a C level MonitoredQueue from PyZMQ which uses a ZeroMQ DEALER socket to perform all load balancing This scheduler does not support any of the advanced features of the Python Scheduler Disabled features when using the ZMQ Scheduler 5 5 The IPython task interface 301 IPython Documentation Release 1 2 1 An Afternoon Hack e Engine unregistration Task farming will be disabled if an engine unregisters Further if an engine is unregistered during computation the scheduler may not recover e Dependencies Since there is no Python logic inside the Scheduler routing decisions cannot be made based on message content e Early destination notification The Python schedulers know which engine gets which task and no tify the Hub This allows graceful handling of Engines coming and going There is no way to know where ZeroMQ messages have gone so there
263. ect request to get basic information about the kernel such as the ports the other ZeroMQ sockets are listening on This allows clients to only have to know about a single port the shell channel to connect to a kernel Message type connect_request 7 2 Messaging in IPython 395 IPython Documentation Release 1 2 1 An Afternoon Hack content Message type connect_reply content shell port int The port the shell ROUTER socket is listening on iopub port ant The port the PUB socket is listening on stdin p rt ant The port the stdin ROUTER socket is listening on Ab port int The port the heartbeat socket is listening on Kernel info If a client needs to know information about the kernel it can make a request of the kernel s information This message can be used to fetch core information of the kernel including language e g Python language version number and Python version number and the Python message spec version number Message type kernel_info_request content Message type kernel_info_reply content Version of messaging protocol mandatory The first integer indicates major version It is incremented when there is any backward incompatible change The second integer indicates minor version It is incremented when there is any backward compatible change protocol_version int int OR SK SK IPytho
264. ed py file after loading a module kills the kernel BUG kernel dies if user sets sys stderr or sys stdout to a file object LaTeX sometimes fails to render in markdown cells with some curly bracket underscore combinations 2898 2902 2899 2890 2892 1340 2881 2751 2879 Skip ipc tests on Windows ActiveState attempt to build ipython 0 12 1 for python 3 2 2 for Mac OS failed Test failure in Python core tests test_magic test_time Test failure when fabric not installed Python tab completion bug for paths Allow input cells to be collapsed command in notebook does not show help in Safari timeit should use minutes to format running time in long running cells When importing a module with a wrong name ipython crashes 62 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 2862 2485 2785 2843 2784 2732 2860 2858 2856 2722 2473 2836 2833 2781 2150 2634 2230 rstrip 2483 2466 2477 2622 2626 1223 2621 2564 2532 2188 2391 2355 2306 2297 timeit should warn of empty contents History navigation breaks in qtconsole gevent input hook Sliently running code in clipboard with paste cpaste and variants Prun t N lt N gt error Test failure with FileLinks class on Windows ipython help notebook gt KeyError KernelManager Where is the installed ipython sc
265. ed in version 0 11 the notebook can now retain 100 of the features of the real IPython But it can also provide the rich media support and high quality Javascript libraries that were not available in browsers even one or two years ago such as high quality mathematical rendering or built in video The notebook has too many useful and important features to describe in these release notes our docu mentation now contains a directory called examples notebooks with several notebooks that illustrate various aspects of the system You should start by reading those named 00_notebook_tour ipynb and 01_notebook_introduction ipynb first and then can proceed to read the others in any order you want To start the notebook server go to a directory containing the notebooks you want to open or where you want to create new ones and type ipython notebook You can see all the relevant options with ipython notebook help ipython notebook help all even more 2 6 0 12 Series 127 IPython Documentation Release 1 2 1 An Afternoon Hack and just like the Qt console you can start the notebook server with pylab support by using ipython notebook pylab for floating matplotlib windows or ipython notebook pylab inline for plotting support with automatically inlined figures Note that it is now possible also to activate pylab support at runtime via Spy lab so you do not need to make this decision w
266. ed into a runtime containment tree not inheritance that i automatically propagates configuration information and ii allows singletons to discover each other in a loosely coupled manner In the future all parts of Python will be subclasses of Configurable All Python developers should become familiar with the config system Created a new Config for holding configuration information This is a dict like class with a few ex tras 1 it supports attribute style access ii it has a merge function that merges two Config instances recursively and iii it will automatically create sub Config instances for attributes that start with an uppercase character Created new configuration loaders in IPython config 1loader These loaders provide a unified loading interface for all configuration information including command line arguments and configura tion files We have two default implementations based on argparse and plain python files These are used to implement the new configuration system Created a top level Application class in IPython core application that is designed to encapsulate the starting of any basic Python program An application loads and merges all the config uration objects constructs the main application configures and initiates logging and creates and configures any Configurable instances and then starts the application running An extended BaselPythonApplication class adds logic for handling the Python directory as well as pro
267. ed print notebook view Improve the design of the tab completion UI Make changes of Application log_format effective Workaround so only one CTRL C is required for a new prompt in gui qt allow formatters to specify metadata improve discovery of public IPs check prefixes for swallowing kernel args Removing old autogrow JS code Update to CodeMirror 3 and start to ship our components 34 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 3229 PR 3228 PR 3226 PR 3225 PR 3222 PR 3223 PR 3089 PR 3221 PR 3205 PR 3088 PR 3201 PR 3220 PR 3011 PR 3217 PR 3174 PR 2979 PR 3215 PR 3181 PR 3200 PR 3186 PR 3212 lation PR 3211 PR 3208 PR 3203 PR 3189 PR 3194 PR 3188 PR 2695 PR 3182 PR 3163 PR 3180 The HTML output type accidentally got removed from the OutputArea Typo in Python Parallel documentation Text in rename dialog was way too big making it lt p gt Removing old restuctured text handler and web service make BlockingKernelClient the default Client add missing mathjax_url to new settings dict add stdin to the notebook Remove references to HTMLCell dead code add ignored args to HasTraits constructor cleanup Python handler settings use much faster regexp for ansi coloring avoid race condition in profile creation IPEP 12 add KernelClient informative error when trying to lo
268. ed to the end of all magics e g autopx_even Z pxresult_even etc For convenience the Client has a activate method as well which creates a DirectView with block True activates it and returns the new View The initial magics registered when you create a client are the result of a call to rc activate with default args 5 4 3 Engines as Kernels Engines are really the same object as the Kernels used elsewhere in Python with the minor exception that engines connect to a controller while regular kernels bind their sockets listening for connections from a QtConsole or other frontends Sometimes for debugging or inspection purposes you would like a QtConsole connected to an engine for more direct interaction You can do this by first instructing the Engine to also bind its kernel to listen for connections In 50 px from IPython parallel import bind_kernel bind_kernel Then if your engines are local you can start a qtconsole right on the engine s In 51 tspx sqtconsole Careful with this one because if your view is of 16 engines it will start 16 QtConsoles 294 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack Or you can view just the connection info and work out the right way to connect to the engines depending on where they live and where you are In 51 px Sconnect_info Parallel execution on engine s a
269. ed version The advan tages of this are e GUIs can be enabled and disabled dynamically at runtime e The active GUI can be switched dynamically at runtime e In some cases multiple GUIs can run simultaneously with no problems e There is a developer API in Python lib inputhook for customizing all of these things For users enabling GUI event loop integration is simple You simple use the gui magic as follows a Sgui GUINAME With no arguments gui removes all GUI support Valid GUINAME arguments are wx qt gtk and tk Thus to use wxPython interactively and create a running wx App object do Sgui wx For information on IPython s matplotlib integration and the matplot1lib mode see this section For developers that want to use Python s GUI event loop integration in the form of a library these capabili ties are exposed in library form in the TPython 1lib inputhook and IPython lib guisupport modules Interested developers should see the module docstrings for more information but there are a few points that should be mentioned here First the PyOSInputHook approach only works in command line settings where readline is activated The integration with various eventloops is handled somewhat differently and more simply when using the standalone kernel as in the qtconsole and notebook Second when using the PyOSInputHook approach a GUI application should not start its event loop Instead all o
270. edits file e 628 import pexpect from Python external in irunner e 596 Irunner e 598 Fix templates for CrashHandler e 590 Desktop e 600 Fix bug with non ascii reprs inside pretty printed lists e 618 1617 e 599 Gui Qt example and docs e 619 manpage update e 582 Updating sympy profile to match the exec_lines of isympy e 578 Check to see if correct source for decorated functions can be displayed e 589 issue 588 e 591 simulate shell expansion on run arguments at least tilde expansion e 576 Show message about paste magic on an IndentationError e 574 Getcwdu e 565 don t move old config files keep nagging the user 166 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 3 9 567 564 e 560 561 556 OL 555 e 552 e 504 551 e 496 550 e 546 e 548 e 542 533 538 e 540 537 536 480 462 534 e 527 e 526 e 528 454 532 e 531 e 525 e S17 Added more docstrings to Python zmq session fix trailing whitespace from reseting indentation Command line args in docs reorder qt support in kernel command line suggestions qt_for_kernel use matplotlib rcParams to decide between PyQt4 and PySide Update usage py to newapp Rm default old config update parallel code for py3k Updating string formatting Make pylab import all configurable Qt editing keybindings Support v2 PyQt4 APIs and PySide
271. ee the installation documentation for directions on how to install the notebook and its dependencies Main features of the web application In browser editing for code with automatic syntax highlighting indentation and tab comple tion introspection The ability to execute code from the browser with the results of computations attached to the code which generated them Displaying the result of computation using rich media representations such as HTML LaTeX PNG SVG etc For example publication quality figures rendered by the matplotlib library can be included inline In browser editing for rich text using the Markdown markup language which can provide commentary for the code is not limited to plain text The ability to easily include mathematical notation within markdown cells using LaTeX and rendered natively by MathJax Notebook documents Notebook documents contains the inputs and outputs of a interactive session as well as additional text that accompanies the code but is not meant for execution In this way notebook files can serve as a complete computational record of a session interleaving executable code with explanatory text mathematics and rich representations of resulting objects These documents are internally JSON files and are saved with the ipynb extension Since JSON is a plain text format they can be version controlled and shared with colleagues Notebooks may be exported to a range of static formats inc
272. efault traceback you may hide important information from the user 6 5 Editor configuration Python can integrate with text editors in a number of different ways e Editors such as X Emacs vim and TextMate can send code to IPython for execution e Python s edit magic command can open an editor of choice to edit a code block The edit command and its alias ed will invoke the editor set in your environment as EDITOR If this variable is not set it will default to vi under Linux Unix and to notepad under Windows You may want to set this variable properly and to a lightweight editor which doesn t take too long to start that is something other than a new instance of Emacs This way you can edit multi line code quickly and with the power of a real editor right inside Python You can also control the editor by setting TerminalInteractiveShell editor in ipython_config py 6 5 Editor configuration 361 IPython Documentation Release 1 2 1 An Afternoon Hack 6 5 1 TextMate Currently TextMate support in Python is broken It used to work well but the code has been moved to TIPython quarantine until it is updated 6 5 2 Vim Paul Ivanov s vim ipython provides powerful Python integration for vim 6 5 3 X Emacs If you are a dedicated Emacs user and want to use Emacs when Python s edit magic command is called you should set up the Emacs server so that new requests are handled by the original process T
273. egqs which uses Matplotlib to plot the result Here is an interactive Python session that uses these functions with SymPy In 7 import sympy In 8 pi sympy pi evalf 40 In 9 pi Out 9 3 141592653589793238462643383279502884197 In 10 pi sympy pi evalf 10000 In 11 digits d for d in str pi 2 create a sequence of digits In 13 freqs one_digit_freqs digits In 14 plot_one_digit_fregqs freqs Out 14 lt matplotlib lines Line2D object at 0x18a55290 gt The resulting plot of the single digit counts shows that each digit occurs approximately 1 000 times but that with only 10 000 digits the statistical fluctuations are still rather large 322 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack Single digit counts in pi 1060 1040 1020 1000 Count 980 960 005 1 2 3 4 5 6 7 8 9 Digit It is clear that to reduce the relative fluctuations in the counts we need to look at many more digits of pi That brings us to the parallel calculation Parallel calculation Calculating many digits of pi is a challenging computational problem in itself Because we want to focus on the distribution of digits in this example we will use pre computed digit of pi from the website of Professor Yasumasa Kanada at the University of Tokyo http www super computing org These digits come in a set of text fi
274. embed for details 4 4 IPython as a system shell 4 4 1 Overview It is possible to adapt Python for system shell usage In the past Python shipped a special sh profile for this purpose but it had been quarantined since 0 11 release and in 1 0 it was removed altogether Neverthe less much of this section relies on machinery which does not require a custom profile You can set up your own sh profile to be different from the default profile such that e Prompt shows the current directory see Prompt customization e Make system commands directly available in alias table by running the srehashx magic If you install new programs along your PATH you might want to run srehashx to update the alias table e turn Zautocall to full mode 4 4 2 Aliases Once you run rehashx all of your PATH has been loaded as IPython aliases so you should be able to type any normal system command and have it executed See alias and tunalias for details on the alias facilities See also Srehashx for details on the mechanism used to load PATH 230 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack 4 4 3 Directory management Since each command passed by ipython to the underlying system is executed in a subshell which exits immediately you can NOT use cd to navigate the filesystem Python provides its own builtin cd magic command to move in the filesystem the is no
275. end freeze on lots of output refocus active cell on dialog close Statelessify mathjaxutils enable comment uncomment selection remove special case in get_home_dir for frozen dists add CONTRIBUTING md use Popen command list for ipexec pylab import adjustments add create Cell and delete Cell js events push cell magic to the head of the transformer line NbConvert Writers No YAML and stuff Pywin32 skips set default client_class for QtKernelManager add strip_encoding_ cookie transformer increase patience for slow kernel startup in tests remove a bunch of unused default_config_file assignments CSS adjustments Don t require HistoryManager to have a shell don t assume tested ipython is on the PATH fix single result AsyncResults Markdown in heading cells take 2 Remove old docs examples catch any exception appending output don t blacklist builtin names Fix frontend deprecation warnings in several examples fix AsyncResult get_dict for single result Fix store magic test Fix config_file_name was ignored Gcf get_active can return None 28 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 3571 PR 3635 PR 3487 PR 3338 PR 3628 PR 3642 PR 3627 PR 3624 PR 3460 PR 3632 PR 3629 PR 3626 PR 3618 PR 3623 PR 3071 PR 3608 PR 3592 PR 3612 PR 3616 PR 3615 PR 3586 PR 3611 PR 3555 PR 3583 PR 3594 PR 3600
276. entation Release 1 2 1 An Afternoon Hack Making patches Overview tell git who you are git config global user email you yourdomain example com git config global user name Your Name Comes Here get the repository if you don t have it git clone git github com ipython ipython git make a branch for your patching cd ipython git branch the fix im thinking of git checkout the fix im thinking of hack hack hack Tell git about any new files you ve made git add somewhere tests test_my_bug py commit work in progress as you go git commit am BF added tests for Funny bug hack hack hack git commit am BF added fix for Funny bug make the patch files git format patch M C master Then send the generated patch files to the ipython mailing list where we will thank you warmly In detail 1 Tell git who you are so it can label the commits you ve made git config global user email you yourdomain example com git config global user name Your Name Comes Here 2 If you don t already have one clone a copy of the ipython repository git clone git github com ipython ipython git cd ipython 3 Make a feature branch This will be where you work on your bug fix It s nice and safe and leaves you with access to an unmodified copy of the code in the main branch git branch the fix im thinking of git checkout t
277. ently than in IPython kernel See our doc on Dependencies for details See also Our Task Interface doc for a simple tutorial with the LoadBalancedView PendingResults to AsyncResults With the departure from Twisted we no longer have the Deferred class for representing unfinished results For this we have an AsyncResult object based on the object of the same name in the built in multiprocessing pool module Our version provides a superset of that interface However unlike in Python kernel we do not have PendingDeferred PendingResult or TaskResult objects Simply this one object the AsyncResult Every asynchronous block False call returns one The basic methods of an AsyncResult are AsyncResult wait timeout wait for the result to arrive AsyncResult get timeout wait for the result to arrive and then return it AsyncResult metadata dict of extra information about execution There are still some things that behave the same as IPython kernel old In 5 pr mec pull a targets 0 1 block False In bo rst Out 6 5 5 new In 5 ar dview pull a targets 0 1 block False In 6 ar r Oubl6 s 5y 5 The r or result property simply calls get waiting for and returning the result See also 5 14 Transitioning from IPython kernel to IPython parallel 345 IPython Documentation Release 1 2 1 An Afternoon Hack AsyncResult details 34
278. er will no longer perform static transformations the processing of escaped commands such as Smagic and system and stripping input prompts from code blocks This functionality was duplicated in IPython core inputsplitter and the latter version was already what Python relied on A new API to transform input will be ready before release Functions from ITPython lib inputhook to control integration with GUI event loops are no longer exposed in the top level of IPython 1lib Code calling these should make sure to import them from IPython 1lib inputhook For all kernel managers the sub_channel1 attribute has been renamed to iopub_channel Users on Python versions before 2 6 6 2 7 1 or 3 2 will now need to call IPython utils doctestreload doctest_reload to make doctests run correctly inside Python Python releases since those versions are unaffected For details see PR 3068 and Python issue 8048 2 2 1 0 Series 9 IPython Documentation Release 1 2 1 An Afternoon Hack e The InteractiveShell cache_main_mod method has been removed and new_main_mod has a different signature expecting a filename where earlier versions ex pected a namespace See PR 3555 for details e The short lived plugin system has been removed Extensions are the way to go NbConvert The major milestone for Python 1 0 is the addition of ITPython nbconvert tools for converting Python notebooks to various other formats Warning n
279. er client to this kernel use IPKernelApp existing kernel 12345 json In this case the IP that you would connect to would still be 127 0 0 1 but you want to specify the additional ssh argument with the hostname of the kernel in this example it s worker user client gt ipython qtconsol ssh worker xisting path to kernel 12349 json Which will write a new connection file with the forwarded ports so you can reuse them IPythonQtConsoleApp To connect another client via this tunnel use IPythonQtConsoleApp existing kernel 12345 ssh json Note again that this opens ports on the client machine that point to your kernel Note the ssh argument is simply passed to openssh so it can be fully specified user host port but it will also respect your aliases etc in ssh config if you have any The second pattern is for connecting to a machine behind a firewall across the internet or otherwise wide network This time we have a machine login that you have ssh access to which can see kernel but client is on another network The important difference now is that client can see login but not worker So we need to forward ports from client to worker via login This means that the kernel must be started listening on external interfaces so that its ports are visible to login user worker gt ipython kernel ip 0 0 0 0 IPKernelApp To connect another client to
280. er function that are defined in the outer function If the names are not in the outer function then there will not be a closure and the generated function will look in globals for the name def g b note that b is not referenced in inner s scope def inner this inner will not have a closure return a 5 13 Details of Parallel Computing with IPython 333 IPython Documentation Release 1 2 1 An Afternoon Hack return inner gl g 1 g2 g 2 gl raises NameError on a a 5 g2 returns 5 gl and g2 will be sendable with IPython and will treat the engine s namespace as globals The pull method is implemented based on this principle If we did not provide pull you could implement it yourself with apply by simply returning objects out of the global namespace In 10 view apply lambda a is equivalent to In 11 view pull a 5 13 2 Running Code There are two principal units of execution in Python strings of Python code e g a 5 and Python functions Python is designed around the use of functions via the core Client method called apply Apply The principal method of remote execution is apply of View objects The Client provides the full ex ecution and communication API for engines via its low level send_apply_message method which is used by all higher level methods of its Views f function The fuction to be called remotel
281. er utilizes the Python client to send Python commands and data through the Python schedulers to the Python engines where those commands are executed and the data processed The design of Python ensures that the client is the only access point for the capabilities of the engines That is the only way of addressing the engines is through a client 5 9 Security details of IPython 311 IPython Documentation Release 1 2 1 An Afternoon Hack A user can utilize the client to instruct the Python engines to execute arbitrary Python commands These Python commands can include calls to the system shell access the filesystem etc as required by the user s application code From this perspective when a user runs an Python engine on a host that engine has the same capabilities and permissions as the user themselves as if they were logged onto the engine s host with a terminal 5 9 3 Secure network connections Overview ZeroMQ provides exactly no security For this reason users of Python must be very careful in managing connections because an open TCP IP socket presents access to arbitrary execution as the user on the engine machines As a result the default behavior of controller processes is to only listen for clients on the loopback interface and the client must establish SSH tunnels to connect to the controller processes Warning If the controller s loopback interface is untrusted then Python should be considered vulner a
282. erface for managing the event loops in their interactive GUI applications Examples can be found in our examples 1ib directory Backwards incompatible changes The Twisted based IPython kernel has been removed and completely rewritten as IPython parallel using ZeroMQ Profiles are now directories Instead of a profile being a single config file profiles are now self contained directories By default profiles get their own Python history log files and everything To create a new profile do ipython profile create lt name gt All Python applications have been rewritten to use KeyValueConfigLoader This means that command line options have changed Now all configurable values are accessible from the command line with the same syntax as in a configuration file The command line options wthread qthread and gthread have been removed Use gui wx gui qt gui gtk instead The extension loading functions have been renamed to load_ipython_extension and unload_ipython_extension e InteractiveShell no longer takes an embedded argument Instead just use the InteractiveShellEmbed class __IPYTHON__ is no longer injected into _ builtin__ Struct __init__ no longer takes None as its first argument It must be a dict or Struct ipmagic has been renamed The functions ipmagic and ipalias have been removed from ___builtins__ The references to the global InteractiveShe
283. ernel handle empty metadata in pyout messages more gracefully my attempt to fix 3735 nbconvert Provide a more useful error for invalid use case nbconvert Allow notebook filenames without their extensions nbconvert Add cwd to default templates search path Update nbconvert docs Nbconvert Export extracted files into nbname_files subdirectory Nicer message when pandoc is missing closes 3730 fix two failing test in Python lib Start what s new for 1 0 PR 3705 Complete rewrite of Python Notebook docs source interactive htmInotebook txt PR 3709 PR 3716 PR 3683 PR 3686 PR 3699 PR 3695 PR 3685 PR 3675 Docs cleanup raw_input fixes for kernel restarts use Y matplotlib in example notebooks remove quarantine svg2pdf unicode fix fix SVG2PDF fix Pager detach document new dependencies documentation 2 3 Issues closed in the 1 0 development cycle 27 IPython Documentation Release 1 2 1 An Afternoon Hack PR 3690 PR 3671 PR 3692 PR 3409 PR 3660 PR 3598 PR 3673 PR 3677 PR 3674 PR 3670 PR 3568 PR 3559 PR 3606 PR 3607 PR 3665 PR 3669 PR 3662 PR 3641 PR 3651 PR 3630 PR 3645 PR 3643 PR 3654 PR 3601 PR 3652 PR 3621 PR 3585 PR 3647 PR 3649 PR 3648 PR 3650 PR 3640 Fixing some css minors in full_html and reveal nbconvert tests Fix rename notebook show error with invalid name Prevent qtconsole front
284. erver login mycluster net After doing this your ipcontroller client json file will look something like this mL Reps st434ar exec_key 9c7779e4 d08a 4c3b ba8e db1f80b562c1 ssh login mycluster net Location lt 10 0 155 Then this file will be all you need for a client to connect to the controller tunneling SSH connections through login mycluster net Database Backend The Hub stores all messages and results passed between Clients and Engines For large and or long running clusters it would be unreasonable to keep all of this information in memory For this reason we have two database backends MongoDB via PyMongo and SQLite with the stdlib sqlite MongoDB is our design target and the dict like model it uses has driven our design As far as we are concerned BSON can be considered essentially the same as JSON adding support for binary data and datetime objects and any new database backend must support the same data types See also MongoDB BSON doc To use one of these backends you must set the HubFactory db_class trait for a simple dict based in memory implementation use dictdb This is the default and the fastest since it doesn t involve the filesystem 276 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack c HubFactory db_class IPython parallel controller dictdb DictDB
285. ery issue and pull request closed on GitHub as well as a full list of individual contributors Major Notebook improvements new user interface and more The Python Notebook which has proven since its release to be wildly popular has seen a massive amount of work in this release cycle leading to a significantly improved user experience as well as many new features The first user visible change is a reorganization of the user interface the left panel has been removed and was replaced by a real menu system and a toolbar with icons Both the tool bar and the header above the menu can be collapsed to leave an unobstructed working area 2 4 0 13 Series 71 IPython Documentation Release 1 2 1 An Afternoon Hack OO wyspectrogram C amp https localhost 8888 899Sfc4a 4 10d 4118 9df8 211274fdce87 Simple spectral analysis An illustration of the Discrete Fourier Transform wI 2s X Dre 0 5 N 1 n 0 In 2 from scipy io import wavfile rate x wavfile read test_mono wav In 5 fig axl ax2 plt subplots 1 2 figsize 16 5 axl plot x axl set_title Raw audio signal ax2 specgram x ax2 set_title Spectrogram Raw audio signal 40000 w IP y Notebook spectrogram Last saved Jun 28 10 35 AM Logout Fie Edit View Insert Cell Kernel Help 8 x 6 t 4 7 gt Code And we can easily view it s spectral structure using matplotlib s builtin specgram routine 30000
286. es Notebook names truncating at the first period second attempt at scrolled long output Cell Worksheet metadata Confirm restart configuration option and checkbox UI qtconsole take prefix into account for completion fix another FreeBSD HOME symlink issue Fix psums example description in docs fix for 1678 undo no longer clears cells avoid duplicate Websockets closed dialog on ws close Support unicode prompts update to latest version of vim ipython Add proc option to script move import RemoteError after get_exc_info Fix for copy action Ctrl C when there is no pager defined in qtconsole 92 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 1948 PR 1942 PR 1940 PR 1938 PR 1925 PR 1936 PR 1937 PR 1935 PR 1849 PR 1759 PR 1931 PR 1933 PR 1932 PR 1928 PR 1923 PR 1926 PR 1893 PR 1921 PR 1919 PR 1918 PR 1917 PR 1913 PR 1911 PR 1909 PR 1907 PR 1906 PR 1845 PR 1902 PR 1900 PR 1899 PR 1898 PR 1897 Fix help string for InteractiveShell ast_node_interactivity swallow stderr of which in utils process find_cmd fix completer css on some Chrome versions remove remaining references to deprecated XREP XREQ names Fix styling of superscripts and subscripts Closes 1924 increase duration of save messages add save f add version checking to pyreadline import test Octave magics g
287. es this attack is not possible Note I may be wrong the unauthorized controller may be easier to fake than this 5 9 5 Other security measures A number of other measures are taken to further limit the security risks involved in running the Python kernel First by default the Python controller listens on random port numbers While this can be overridden by the user in the default configuration an attacker would have to do a port scan to even find a controller to attack When coupled with the relatively short running time of a typical controller on the order of hours an attacker would have to work extremely hard and extremely fast to even find a running controller to attack Second much of the time especially when run on supercomputers or clusters the controller is running behind a firewall Thus for engines or client to connect to the controller e The different processes have to all be behind the firewall or e The user has to use SSH port forwarding to tunnel the connections through the firewall In either case an attacker is presented with additional barriers that prevent attacking or even probing the system 5 9 6 Summary IPython s architecture has been carefully designed with security in mind The capabilities based authentica tion model in conjunction with SSH tunneled TCP IP channels address the core potential vulnerabilities in the system while still enabling user s to use the system in open networks
288. es in inputhook 1462 expedite parallel tests 1418 Strict mode in javascript 1410 Add javascript library and css stylesheet loading to JS class 1427 922 again 1448 Fix for 875 Never build unicode error messages 1458 use eval to uncan References 1455 Python3 install fails 1450 load mathjax from CDN via https 1182 Qtconsole multiwindow 1439 Notebook not storing heading celltype information 1451 include heading level in JSON 1444 Fix pyhton gt python typos 1412 Input parsing issue with prun 1414 ignore errors in shell var_expand 1441 1 Enable Python notebook kernel execute to publish display_ even it is not called with a code cell and 2 remove empty html element when execute display_ 1431 Beginner Error ipython qtconsole 1436 ipython qtconsole gui qt hangs on 64 bit win7 1433 websocket connection fails on Chrome 1430 Fix for tornado check for tornado lt 1 1 0 120 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 1408 test_get_home_dir_3 failed on Mac OS X 1413 get_home_dir expands symlinks adjust test accordingly 1420 fixes 922 823 KnownFailure tests appearing as errors 1385 updated and prettified magic doc strings 1406 Browser selection 1411 ipcluster starts 8 engines successfully but Client only finds two 1375 history g f file encoding issue 1377 Sav
289. est button 4 Admin lt gt Unwatch Ly Pull Request LZ Downl Downloads 0 Wiki 1 Graphs Enter a message we suggest you select only ipython as the recipient The message will go to the ipython mailing list Please feel free to add others from the list as you like Merging from trunk This updates your code from the upstream ipython github repo Overview go to your master branch git checkout master pull changes from github git fetch upstream merge from upstream git merge upstream master In detail We suggest that you do this only for your master branch and leave your feature branches unmerged to keep their history as clean as possible This makes code review easier git checkout master Make sure you have done Linking your repository to the upstream repo Merge the upstream code into your current development by first pulling the upstream repo to a copy on your local machine git fetch upstream then merging into your current branch git merge upstream master 380 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack Deleting a branch on github git checkout master delete branch locally git branch D my unwanted branch delete branch on github git push origin my unwanted branch Note the colon before test branch See also http github com guides remove a remote branch
290. est on WinXP wait for resource cleanup BUG test runner fails in Windows if filenames contain spaces Fix home path expansion test in Windows 2 5 Issues closed in the 0 13 development cycle 91 IPython Documentation Release 1 2 1 An Afternoon Hack PR 2021 PR 2022 PR 1996 PR 2014 PR 2012 PR 2004 PR 2010 PR 2002 PR 1999 PR 1981 PR 1994 PR 1951 PR 1974 PR 1964 PR 1991 PR 1988 PR 1979 PR 1970 PR 1978 PR 1825 PR 1934 PR 1746 PR 1944 PR 1973 PR 1967 PR 1965 PR 1952 PR 1962 PR 1955 PR 1945 PR 1956 PR 1950 Fix Windows pathname issue in odd encoding test don t check writability in test for get_home_dir when HOME is undefined frontend test tweaks relax profile regex in notebook Mono cursor offset Clarify generic message spec vs Python message API in docs notebook Print a warning but do not abort if no webbrowser can be found Refactor magic into a lsmagic_docs API function Yomagic help display line and cell magics in alphabetical order Clean BG processes created by script on kernel exit Fix RST misformatting minor notebook startup notebook dir adjustments Allow path completion on notebook allow multiple instances of a Magic fix _ofind attr in page check for active frontend in update_restart_checkbox Add support for tox http tox testrun org and Travis CI http travis ci org dblclick to restore size of imag
291. esult 338 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack represents multiple calls e g any time targets is plural then a CompositeError a subclass of RemoteError will be raised See also For more information on remote exceptions see the section in the Direct Interface Extended interface Other extensions of the AsyncResult interface include convenience wrappers for get AsyncResults have a property result with the short alias r which simply call get Since our object is designed for representing parallel results it is expected that many calls any of those submitted via DirectView will map results to engine IDs We provide a get_dict which is also a wrapper on get which returns a dictionary of the individual results keyed by engine ID You can also prevent a submitted job from actually executing via the AsyncResult s abort method This will instruct engines to not execute the job when it arrives The larger extension of the AsyncResult API is the metadata attribute The metadata is a dictionary with attribute access that contains logically enough metadata about the execution Metadata keys timestamps submitted When the task left the Client started When the task started execution on the engine completed When execution finished on the engine received When the result arrived on the Client note that it is not known when the result a
292. et in the heartbeat process This can be a uuid or even a full message but there doesn t seem to be a need for packing up a message when the sender and receiver are the exact same Python object The model is this monitor send str self lifetime 1 2345678910 and the monitor receives some number of messages of the form uuid abcd dead beef 1 2345678910 where the first part is the zmq IDENTITY of the heart s DEALER on the engine and the rest is the message sent by the monitor No Python code ever has any access to the message between the monitor s send and the monitor s recv 7 2 9 ToDo Missing things include e Important finish thinking through the payload concept and API e Important ensure that we have a good solution for magics like edit It s likely that with the payload concept we can build a full solution but not 100 clear yet e Finishing the details of the heartbeat protocol e Signal handling specify what kind of information kernel should broadcast or not when it receives signals 7 2 Messaging in IPython 401 IPython Documentation Release 1 2 1 An Afternoon Hack 7 3 Messaging for Parallel Computing This is an extension of the messaging doc Diagrams of the connections can be found in the parallel con nections doc ZMQ messaging is also used in the parallel computing Python system All messages to from kernels remain the same as the single
293. etadata should be unicode not bytes PR 4232 no highlight if no language specified PR 4218 Fix display of SyntaxError when py file is modified PR 4217 avoid importing numpy at the module level PR 4213 fixed dead link in examples notebooks readme to Part 3 14 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 4183 ESC should be handled by CM if tooltip is not on PR 4193 Update for 3549 Append Firefox overflow x fix PR 4205 use TextlOWrapper when communicating with pandoc subprocess PR 4204 remove some extraneous print statements from Python parallel PR 4201 HeadingCells cannot be split or merged Issues 17 1 2 1 1 2 0 5101 4892 4810 4765 4761 4721 4602 3386 API 4485 4351 4334 4039 4256 4122 3819 4203 4200 IPython 1 2 0 notebook fail with 500 Internal Server Error IPython qt test failure with python3 4 ipcluster bug in clean_logs flag missing build script for highlight js ipv6 address triggers cookie exception purge_results with jobid crashing looking for insight ipcluster stop fails after ipcluster start daemonize using python3 3 Magic paste not working in Python 3 3 2 TypeError Type str doesn t support the buffer Incorrect info in Messaging in IPython documentation parallel apps launcher py error NotebookApp webapp_settings static_url_prefix ca
294. ets the detail of the logging level during the startup of ipython The default is 30 and the possible values are 0 10 20 30 40 50 Higher is quieter and lower is more verbose This can also be set by the name of the logging level e g INFO 20 WARN 30 Some options such as extensions and startup code can be set for any application that starts an InteractiveShell These apps are subclasses of InteractiveShellApp Since subclasses in herit configuration setting a trait of c InteractiveShellApp will affect all Python applications but if you want terminal Python and the QtConsole to have different values you can set them via c TerminallIPythonApp and c IPKernelApp respectively c InteractiveShellApp extensions A list of strings each of which is an importable Python extension See Python extensions for more details about extensions c InteractiveShellApp exec_lines A list of strings each of which is Python code that is run in the user s namespace after Python start These lines can contain full Python syntax with magics etc c InteractiveShellApp exec_files A list of strings each of which is the full pathname of a py or ipy file that will be executed as Python starts These files are run in Python in the user s namespace Files with a py extension need to be pure Python Files with a ipy extension can have custom Python syntax magics etc These files need to be in the cwd the ipythondir or be absolute pat
295. f this is handled by the PyOSInputHook This means that applications that are meant to be used both in Python and as standalone apps need to have special code to detects how the application is being run We highly recommend using Python s support for this Since the details vary slightly between toolkits we point you to the various examples in our source directory examples 1ib that demonstrate these capabilities Third unlike previous versions of Python we no longer hijack replace them with no ops the event loops This is done to allow applications that actually need to run the real event loops to do so This is often needed to process pending events at critical points Finally we also have a number of examples in our source directory examples 1ib that demonstrate these capabilities 4 3 IPython reference 227 IPython Documentation Release 1 2 1 An Afternoon Hack PyQt and PySide When you use gui qt or matplot1lib qt Python can work with either PyQt4 or PySide There are three options for configuration here because PyQt4 has two APIs for QString and QVariant v1 which is the default on Python 2 and the more natural v2 which is the only API supported by PySide v2 is also the default for PyQt4 on Python 3 IPython s code for the QtConsole uses v2 but you can still use any interface in your code since the Qt frontend is in a different process The default will be to import PyQt4 without configurat
296. fault and will be located in lt IPYTHONDIR gt profile_default The next thing you need to know is what to call your configuration file The basic idea is that each ap plication has its own default configuration filename The default named used by the ipython command line program is ipython_config py and all Python applications will use this file Other applica tions such as the parallel ipcluster scripts or the QtConsole will load their own config files after ipython_config py To load a particular configuration file instead of the default the name can be overridden by the config_file command line flag To generate the default configuration files do ipython profile create and you will have a default ipython_config py in your Python directory under profile_default If you want the default config files for the IPython parallel applica tions add parallel to the end of the command line args 6 1 Overview of the IPython configuration system 351 IPython Documentation Release 1 2 1 An Afternoon Hack Locating these files From the command line you can quickly locate the IPYTHONDIR or a specific profile with S ipython locate home you ipython S ipython locate profile foo home you ipython profile_foo These map to the utility functions IPython utils path get_ipython_dir and IPython utils path locate_profile respectively 6 1 6 Profiles A profile is a directory containi
297. ficient contributions are made that im prove 0 10 1 we will roll them into future releases For this purpose we will have a branch called 0 10 2 on github on which you can base your contributions For this release we applied approximately 60 commits totaling a diff of over 7000 lines 0 10 1 amirbar dist gt git diff oneline rel 0 10 we 1 7296 Highlights of this release e The only significant new feature is that Python s parallel computing machinery now supports natively the Sun Grid Engine and LSF schedulers This work was a joint contribution from Justin Riley Satra Ghosh and Matthieu Brucher who put a lot of work into it We also improved traceback handling in remote tasks as well as providing better control for remote task IDs New Python Sphinx directive contributed by John Hunter You can use this directive to mark blocks in reSructuredText documents as containing Python syntax including figures and the will be executed during the build In 2 plt figure nsure a fresh figure savefig psimple png width 4in In 3 plt plot 1 2 3 Out 3 lt matplotlib lines Line2D object at 0x9b74d8c gt e Various fixes to the standalone ipython wx application 2 10 0 10 series 183 IPython Documentation Release 1 2 1 An Afternoon Hack We now ship internally the excellent argparse library graciously licensed under BSD terms by Steven Bethard Now 2010 th
298. fig py 4 4 6 String lists String lists Python utils text SList are handy way to process output from system commands They are produced by var cmd syntax First we acquire the output of Is l Q doc examples 2 gt lines 1s 1 totak 23 rw rw rw 1 ville None 1163 Sep 30 2006 example demo py 4 4 IPython as a system shell 231 IPython Documentation Release 1 2 1 An Afternoon Hack rw rw rw 1 ville None 1927 Sep 30 2006 exampl mbed short py rwxrwxrwx 1 ville None 4606 Sep 1 17 15 example embed py rwxrwxrwx 1 ville None 1017 Sep 30 2006 example gnuplot py rwxrwxrwx 1 ville None 339 Jun 11 18 01 extension py rwxrwxrwx 1 ville None 113 Dec 20 2006 seteditor py rwxrwxrwx 1 ville None 245 Dec 12 2006 seteditor pyc Now let s take a look at the contents of lines the first number is the list element number Q doc examples 3 gt lines lt 3 gt SList p n l s grep fields available Value 0 total 23 1 rw rw rw 1 ville None 1163 Sep 30 2006 example demo py 2 rw rw rw 1 ville None 1927 Sep 30 2006 exampl mbed short py 3 rwxrwxrwx 1 ville None 4606 Sep 1 17 15 example embed py 4 rwxrwxrwx 1 ville None 1017 Sep 30 2006 example gnuplot py 5 rwxrwxrwx 1 ville None 339 Jun 11 18 01 extension py 6 rwxrwxrwx 1 ville None 113 Dec 20 2006 seteditor py 7 rwxrwxrwx 1 ville None 2
299. foo The current behaviour is closer to the documented behaviour and should not trip anyone 2 12 4 Older releases Changes in earlier releases of Python are described in the older file ChangeLog Please refer to this document for details 2 12 0 8 series 193 IPython Documentation Release 1 2 1 An Afternoon Hack 194 Chapter 2 What s new in IPython CHAPTER 3 Installation Python requires Python 2 6 2 7 or 3 2 Note If you need to use Python 2 5 you can find an old version 0 10 of Python here 3 1 Quickstart If you have setuptools the quickest way to get up and running with Python is easy_install ipython all This will download and install Python and its main optional dependencies e jinja2 needed for the notebook sphinx needed for nbconvert pyzmg needed for IPython s parallel computing features qt console and notebook pygments used by nbconvert and the Qt console for syntax highlighting tornado needed by the web based notebook nose used by the test suite readline on OS X or pyreadline on Windows needed for the terminal To run Python s test suite use the iptest command S iptest Note S pip install ipython all will also work in many cases but it will ignore the binary eggs of packages such as pyzmq and readline which may be required for some users on Windows or OS X 195 IPython Documentation
300. fore the reload causes the new code for foo to be executed Some of the known remaining caveats are e Replacing code objects does not always succeed changing a property in a class to an ordinary method or a method to a member variable can cause problems but in old objects only e Functions that are removed eg via monkey patching from a module before it is reloaded are not upgraded e C extension modules cannot be reloaded and so cannot be autoreloaded 6 2 IPython extensions 357 IPython Documentation Release 1 2 1 An Afternoon Hack cythonmagic octavemagic rmagic storemagic store magic for lightweight persistence Stores variables aliases and macros in Python s database To automatically restore stored variables at startup add this to your ipython_config py file c StoreMagic autorestore Tru sympyprinting A print function that pretty prints sympy Basic objects moduleauthor Brian Granger Usage Once the extension is loaded Sympy Basic objects are automatically pretty printed As of SymPy 0 7 2 maintenance of this extension has moved to SymPy under sympy interactive ipythonprinting any modifications to account for changes to SymPy should be submitted to SymPy rather than changed here This module is maintained here for backwards compatablitiy with old SymPy versions 6 3 Configuring the ipython command line application This section contains information about how to confi
301. full support for the Windows HPC job scheduler meaning that ipcluster can use this job scheduler to start the controller and engines In our experience the Windows HPC job scheduler is particularly well suited for interactive applications such as Python Once ipcluster is configured properly a user can start an Python cluster from their local workstation almost instantly without having to log on to the head node as is typically required by Unix based job schedulers This enables a user to move seamlessly between serial and parallel computations In this section we show how to use ipcluster to start an Python cluster using the Windows HPC Server 2008 job scheduler To make sure that ipcluster is installed and working properly you should first try to start an Python cluster on your local host To do this open a Windows Command Prompt and type the following command ipcluster start n 2 You should see a number of messages printed to the screen The result should look something like this Microsoft Windows Version 6 1 7600 Copyright c 2009 Microsoft Corporation All rights reserved Z gt ipcluster start profile mycluster 5 10 Getting started with Windows HPC Server 2008 317 IPython Documentation Release 1 2 1 An Afternoon Hack PClusterStart Using existing profile dir u blue domainusers bgrangey ipython PClusterStart tarting ipcluster with daemon False PClusterStart reati
302. functions anyway you can just use the builtin map with a RemoteFunction or a DirectView s map method In 62 serial_result map lambda x xx x10 range 32 In 63 parallel_result dview map_sync lambda x x 10 range 32 In 67 serial_result parallel_result Out 67 True Note The DirectView s version of map does not do dynamic load balancing For a load balanced version use a LOadBalancedView See also map is implemented via ParallelFunction 5 3 IPython s Direct interface 279 IPython Documentation Release 1 2 1 An Afternoon Hack Remote function decorators Remote functions are just like normal functions but when they are called they execute on one or more engines rather than locally Python provides two decorators In 10 dview remote block True def getpid rires import os PEEN return os getpid In 11 getpid Out 11 12345 12346 12347 12348 The parallel decorator creates parallel functions that break up an element wise operations and dis tribute them reconstructing the result In 12 import numpy as np In 13 A np random random 64 48 In 14 dview parallel block True def pmul A B ee return AxB In 15 C_local A A In 16 C_remote pmul A A In 17 C_local C_remote all Out 17 True Calling a parallel function does not correspond to map It is used
303. g GUI event loop integration gthread qthread q4thread wthread pylab As of Python version 0 11 these have been removed Please see the new gui magic command or this section for details on the new interface or specify the gui at the commandline ipython gui qt Command line Options To see the options Python accepts use ipython help and you probably should run the output through a pager such as ipython help less for more convenient reading This shows all the options that have a single word alias to control them but Python lets you configure all of its objects from the command line by passing the full class name and a corresponding value type ipython help all to see this full list For example ipython matplotlib qt is equivalent to 208 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack ipython TerminalIPythonApp matplotlib qt Note that in the second form you must use the equal sign as the expression is evaluated as an actual Python assignment While in the above example the short form is more convenient only the most common options have a short form while any configurable variable in Python can be set at the command line by using the long form This long form is the same syntax used in the configuration files if you want to set these options permanently 4 3 2 Interactive use Python is m
304. g dict Don t treat bytes objects as json safe DOC msg msg_type removed fix colors lt color gt option in py python command args fix colors lt color gt option in py python command args Run PyOs_InputHook in pager to keep plot windows interactive Remove ipythonrc references from documentation Tiny doc fixes ZMQ terminal frontend Allow magic argument filenames with spaces to be specified with quotes under win32 respect encoding of display data from urls doc improvements for running notebook via secure protocol use null char to start markdown cell placeholder Minor fixes to the htmlnotebook use bundled argparse if system argparse is lt 1 1 Htmlnotebook Add import time to Python parallel apps launcher py as time sleep is called without time being imported 714 718 717 716 Install mathjax for offline use Underline keyboard shortcut characters on appropriate buttons Add source highlighting to markdown snippets update EvalFormatter to allow arbitrary expressions 2 7 Issues closed in the 0 12 development cycle 143 IPython Documentation Release 1 2 1 An Afternoon Hack 712 713 709 706 674 679 696 686 685 384 647 587 703 699 666 683 689 702 701 670 579 687 627 646 660 659 Reset execution counter after cache is cleared Align colons in html notebook help dialog Allow usage of in notebook names Implement stat
305. g messages Note that this polling does not actually make any network requests It simply performs a select operation to check if messages are already in local memory waiting to be handled The method that handles incoming messages is spin This method flushes any waiting messages on the various incoming sockets and updates the state of the Client If you need to wait for particular results to finish you can use the wait method which will call spin until the messages are no longer outstanding Anything that represents a collection of messages such as a list of msg_ids or one or more AsyncResult objects can be passed as argument to wait A timeout can be specified which will prevent the call from blocking for more than a specified time but the default behavior is to wait forever The client also has an out standing attribute a set of msg_ids that are awaiting replies This is the default if wait is called with no arguments i e wait on all outstanding messages Note TODO wait example 5 13 9 Map Many parallel computing problems can be expressed as a map or running a single program with a variety of different inputs Python has a built in map which does exactly this and many parallel execution tools in Python such as the built in multiprocessing Pool object provide implementations of map All View objects provide a map method as well but the load balanced and direct implementations differ Views map met
306. g on python3 Fix for pdef on Python 3 Failing test that prun does not clobber string escapes Fix for 1428 temporarily skip autoreload tests autoreload extension crashes ipython find_file errors on windows Fix for 1908 use os path normcase for safe filename comparisons py3compat fixes for script and tests px doesn t work shows info for px general cell magic problem ofind finds non unique cell magics Win64 binary install fails Source file not found for magics Fixes to inspection machinery for magics 2 5 Issues closed in the 0 13 development cycle 107 IPython Documentation Release 1 2 1 An Afternoon Hack 1774 Some magics seems broken 1586 Clean up tight coupling between Notebook CodeCell and Kernel Javascript objects 1632 Win32 shell interactivity apparently broke qtconsole cd magic 1902 Workaround fix for gh 1632 minimal revert of gh 1424 1900 Cython libs 1503 Cursor is offset in notebook in Chrome 17 on Linux 1426 Qt console doesn t handle the gui flag correctly 1180 Can t start Python kernel in Spyder 581 test Python zmq 1593 Name embedded in notebook overrides filename 1899 add ScriptMagics to class list for generated config 1618 generate or minimize manpages 1898 minimize manpages 1896 Windows apparently spurious warning Excluding nonexistent file test_exampleip 1897 use glob for bad exclusion warning 1215 upda
307. g sample file illustrating how to use the embedding functionality is provided in the examples directory as example embed py It should be fairly self explanatory usr bin env python An example of how to embed an IPython shell into a running program Please see the documentation in the IPython Shell module for more details The accompanying fil xampl mbed short py has quick code fragments for embedding which you can cut and paste in your code once you understand how things work The code in this file is deliberately extra verbose meant for learning from future import print_function The basics to get you going IPython sets the __IPYTHON__ variable so you can know if you have nested copies running Try running this code both at the command line and from inside IPython with run example embed py from IPython config loader import Config try get_ipython 4 3 IPython reference 221 IPython Documentation Release 1 2 1 An Afternoon Hack except NameError nested 0 cfg Config prompt_config prompt_config prompt_config prompt_config cfg PromptManager in_template In lt gt in2_template INNOD out_template Out lt gt else print Running nested copies of IPython print The prompts for the nested copy have been modified cfg Config nested i First import th mbeddable shell class from IPython terminal embed import Interac
308. ges that do not contain this key are acted upon in any way The key itself is never sent over the network There is exactly one shared key per cluster it must be the same everywhere Typically the controller creates this key and stores it in the private connection files ipython enginelclient json These files are typically stored in the ipython profile_ lt name gt security directory and are maintained as readable only by the owner just as is common practice with a user s keys in their ssh directory 312 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack Warning It is important to note that the signatures protect against unauthorized messages but as there is no encryption provide exactly no protection of data privacy It is possible however to use a cus tom serialization scheme via Session packer unpacker traits that does incorporate your own encryption scheme 5 9 4 Specific security vulnerabilities There are a number of potential security vulnerabilities present in Python s architecture In this section we discuss those vulnerabilities and detail how the security architecture described above prevents them from being exploited Unauthorized clients The Python client can instruct the Python engines to execute arbitrary Python code with the permissions of the user who started the engines If an attacker were able to connect their own hostile Python clie
309. ges to hang race conditions in profile directory creation OverflowError execption in handlers py core profiledir check_startup_dir doesn t work inside py2exe d installation Feature folders for ipython notebook dashboard cell magics do not work with empty lines after 2447 Default plot colors unsuitable for red green colorblind users n foo turns into n foo in triple quoted strings File cell magic fails with blank lines ocython a stopped working obsolete imports in import autocompletion Python2 Unhandled exception __builtin__ True False script magic error message loop 2 3 Issues closed in the 1 0 development cycle 59 IPython Documentation Release 1 2 1 An Afternoon Hack 3009 3059 3104 3176 1133 2975 3017 31061 2835 2628 2612 use XDG_CACHE_HOME for cython objects Bugs in 00_notebook_tour example Integrate a javascript file manager into the notebook front end Particular equation not rendering notebook notebook readonly and upload files UI notebook python file and cell toolbar SciPy weave broken in Python notebook qtconsole paste macro not reading spaces correctly paste not working on WinXpSP3 ipython 0 13 1 py2 win32 PROPER exe python27 Make transformers work for lines following decorators Multiline String containing n foon confuses interpreter to replace foo with get_ipython magic w pinfo foo 2539 2507 2164 3106 2438
310. gine_set_launcher_class and IPClusterStart controller_launcher_class configurables which can be the fully specified object name e g TPython parallel apps launcher LocalControllerLauncher but if you are using Python s builtin launchers you can specify just the class name or even just the prefix e g c IPClusterEngines engine_launcher_class SSH equivalent to c IPClusterEngines engine_launcher_class SSHEngineSetLauncher both of which expand to c IPClusterEngines engine_launcher_class IPython parallel apps launcher SSHE The shortest form being of particular use on the command line where all you need to do to get an Python cluster running with engines started with MPI is gt ipcluster start engines MPI Assuming that the default MPI config is sufficient Note shortcuts for builtin launcher names were added in 0 12 as was the _class suffix on the config urable names If you use the old 0 11 names e g engine_set_launcher they will still work but you will get a deprecation warning that the name has changed Note The Launchers and configuration are designed in such a way that advanced users can subclass and configure them to fit their own system that we have not yet supported such as Condor Using ipcluster in mpiexec mpirun mode The mpiexec mpirun mode is useful if you 1 Have MPI installed 2 Your systems are
311. gs if any varargs str The name of the varkw kw if any varkw str The values as strings of all default arguments Note that these must be matched in reverse with the args list above since the first positional args have no default value at all defaults list SO OR OH OR For instances provide the constructor signature the definition of the __init__ method init definition s Str Docstrings for any object function method module package with a docstring we show it But in addition we may provide additional docstrings For example for instances we will show the constructor and class docstrings as well if available docstring str For instances provide the constructor and class docstrings rinit dogstring str Yelass _docstring str If it s a callable object whose call method has a separate docstring and definition line Callide t Str Gall docstring lt Sti If detail_level was 1 we also try to find the source code that defines the object if possible The string None will indicate that no source was found source str 7 2 Messaging in IPython 393 IPython Documentation Release 1 2 1 An Afternoon Hack Complete Message type complete_request content The text to be completed such as a is this may be an empty string if the frontend does not do any lexing in which case the
312. gt Paul Ivanov lt pivanov314 at gmail com gt Pauli Virtanen lt pav at iki fi gt Pete Aykroyd lt aykroyd at gmail com gt Prabhu Ramachandran lt prabhu at enthought com gt Puneeth Chaganti lt punchagan at gmail com gt Robert Kern lt robert kern at gmail com gt Satrajit Ghosh lt satra at mit edu gt Stefan van der Walt lt stefan at sun ac za gt Szabolcs Horvat lt szhorvat at gmail com gt Thomas Kluyver lt takowl at gmail com gt Thomas Spura lt thomas spura at gmail com gt Timo Paulssen lt timonator at perpetuum immobile de gt Valentin Haenel lt valentin haenel at gmx de gt Yaroslav Halchenko lt debian at onerussian com gt Note This list was generated with the output of git log rel 0 11 HEAD format aN lt SaE gt sed s at sed s lt gt sort u after some cleanup If you should be on this list please add yourself 2 6 0 12 Series 133 IPython Documentation Release 1 2 1 An Afternoon Hack 2 7 Issues closed in the 0 12 development cycle 2 7 1 Issues GitHub stats for closed in 0 12 1 bugfix release 0 12 1 12 28 2011 04 16 2012 backporting pull requests from 0 13 We closed a total of 71 issues 44 pull requests and 27 issues this is the full list generated with the script tools github_stats py This list is automatically generated and may be incomplete Pull Requests 44 PR 1175 PR 1213 string PR
313. guage associated to a code cell is Python but other languages such as Julia and R can be handled using cell magic commands When a code cell is executed code that it contains is sent to the kernel associated with the notebook The results that are returned from this computation are then displayed in the notebook as the cell s output The output is not limited to text with many other possible forms of output are also possible including matplotlib figures and HTML tables as used for example in the pandas data analysis package This is known as Python s rich display capability See also Basic Output example notebook Rich Display System example notebook 4 6 The IPython Notebook 247 IPython Documentation Release 1 2 1 An Afternoon Hack Markdown cells You can document the computational process in a literate way alternating descriptive text with code using rich text In Python this is accomplished by marking up text with the Markdown language The correspond ing cells are called Markdown cells The Markdown language provides a simple way to perform this text markup that is to specify which parts of the text should be emphasized italics bold form lists etc When a Markdown cell is executed the Markdown code is converted into the corresponding formatted rich text Markdown allows arbitrary HTML code for formatting Within Markdown cells you can also include mathematics in a straightforward way using standard L
314. gurable in ipython_config py or at the command line as in instruct IPython to sign messages with that key instead of a new UUID gt ipython qtconsole Session keyfile SIPYTHONDIR sessionkey SSH Tunnels Sometimes you want to connect to machines across the internet or just across a LAN that either doesn t permit open ports or you don t trust the other machines on the network To do this you can use SSH tunnels SSH tunnels are a way to securely forward ports on your local machine to ports on another machine to which you have SSH access In simple cases Python s tools can forward ports over ssh by simply adding the ssh remote argument to the usual existing set of flags for connecting to a running kernel after copying the JSON connection file or its contents to the second computer Warning Using SSH tunnels does not increase localhost security In fact when tunneling from one machine to another both machines have open ports on localhost available for connections to the kernel There are two primary models for using SSH tunnels with Python The first is to have the Kernel listen only on localhost and connect to it from another machine on the same LAN First let s start a kernel on machine worker listening only on loopback 4 5 A Qt Console for Python 241 IPython Documentation Release 1 2 1 An Afternoon Hack user worker gt ipython kernel IPKernelApp To connect anoth
315. gure the ipython command line application See the configuration overview for a more general description of the configuration system and configuration file format The default configuration file for the aipython command line application is profile_default ipython_config py in your Python directory By setting the attributes in this file you can configure the application To create the default config file run this command ipython profile create Most configuration attributes that this file accepts are associated with classes that are subclasses of Configurable Applications themselves are Configurable as well so we will start with some application level config 358 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack 6 3 1 Application level configuration Assuming that your configuration file has the following at the top c get_config the following attributes are set application wide terminal Python only flags c TerminalIPythonApp display banner A boolean that determined if the banner is printer when ipython is started c TerminalIPythonApp classic A boolean that determines if Python starts in classic mode In this mode the prompts and everything mimic that of the normal python shell c TerminalIPythonApp nosep A boolean that determines if there should be no blank lines between prompts c Application log_level An integer that s
316. h Fork tu Pull Request Issues 0 Downloads 0 Wiki 1 Graphs Now after a short pause and some Hardcore forking action you should find yourself at the home page for your own forked copy of ipython Set up your fork First you follow the instructions for Making your own copy fork of ipython Overview git clone git github com your user name ipython git cd ipython git remote add upstream git github com ipython ipython git In detail Clone your fork 1 Clone your fork to the local computer with git clone git github com your user name ipython git 2 Investigate Change directory to your new repo cd ipython Then git branch a to show you all branches You ll get something like x master remotes origin master 7 1 Working with IPython source code 375 IPython Documentation Release 1 2 1 An Afternoon Hack This tells you that you are currently on the master branch and that you also have a remote con nection to origin master What remote repository is remote origin Try git remote v to see the URLs for the remote They will point to your github fork Now you want to connect to the upstream ipython github repository so you can merge in changes from trunk Linking your repository to the upstream repo cd ipython git remote add upstream git github com ipython ipython git upstream here is just the arbitrary name we re using to refer to the main ipython reposito
317. h AsyncResult These objects store a variety of metadata about each task including various timestamps We can validate that the dependencies were respected by checking that each task was started after all of its predecessors were completed def validate_tree G results Validate that jobs executed after their dependencies for node in G started results node metadata started for parent in G predecessors node finished results parent metadata completed assert started gt finished s should have happened after s noda We can also validate the graph visually By drawing the graph with each node s x position as its start time all arrows must be pointing to the right if dependencies were respected For spreading the y position will be the runtime of the task so long tasks will be at the top and quick small tasks will be at the bottom In 10 from matplotlib dates import date2num In 11 from matplotlib cm import gist_rainbow In 12 pos colors In 12 for node in G dees md results node metadata AEE start date2num md started bie cae runtime date2num md completed start area are pos node start runtime 330 Chapter 5 Using IPython for parallel computing parent IPython Documentation Release 1 2 1 An Afternoon Hack sema colors node md engine_id In 13 nx draw G pos node_list colors keys node_color colors va
318. h iopub with AsyncResults Fix longest_substr gt fix mpr again Document globbing in history g lt pattern gt fix html in notebook example Fix regression in embed from pull request 2096 track sha of master in test_pr messages Fast tests add data publication message minor test_pr tweaks Improve Image format validation and add html width height Reapply monkeypatch to inspect findsource remove spurious print statement from setupbase py adjust how canning deals with import strings fix css typo 46 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 2223 PR 2214 PR 2199 PR 2212 PR 2190 PR 2200 PR 2194 PR 2198 PR 2175 PR 1990 PR 2069 PR 2202 PR 2204 PR 2148 PR 2179 PR 2124 PR 2169 PR 2174 PR 1930 PR 2189 PR 2186 PR 2183 PR 2185 PR 2138 PR 2182 PR 1831 PR 2051 PR 1471 PR 2181 PR 2116 PR 2092 PR 2159 Custom tracebacks use KernelApp exec_lines files in IPEngineApp Wrap JS published by javascript in try catch catch errors in markdown javascript Update code mirror 2 22 to 2 32 documentation build broken in bb429da5b clean nan inf in json_clean fix mpr for earlier git version add FileFindHandler for Notebook static files can func_defaults start improving serialization in parallel code Create a unique amp temporary IPYTHONDIR for each testing group
319. handling Python exceptions raised in parallel execution are gathered and presented to the top level code For more information see our overview of using Python for parallel computing 1 4 1 Portability and Python requirements As of the 1 0 release Python works with Python 2 6 2 7 3 2 and 3 3 Version 0 12 introduced full support for Python 3 Version 0 11 worked with Python 2 6 and 2 7 only Versions 0 9 and 0 10 worked with Python 2 4 and above not including Python 3 Python is known to work on the following operating systems e Linux e Most other Unix like OSs AIX Solaris BSD etc e Mac OS X e Windows CygWin XP Vista etc See here for instructions on how to install Python 1 4 Interactive parallel computing 5 IPython Documentation Release 1 2 1 An Afternoon Hack 6 Chapter 1 Introduction CHAPTER 2 What s new in IPython This section documents the changes that have been made in various versions of Python Users should consult these pages to learn about new features bug fixes and backwards incompatibilities Developers should summarize the development work they do here in a user friendly format 2 1 Development version This document describes in flight development work 2 1 1 Backwards incompatible changes 2 2 1 0 Series 2 2 1 Release 1 0 0 An Afternoon Hack Python 1 0 requires Python 2 6 5 or 3 2 1 It does not support Python 3 0 3 1 or 2 5 This is a big release
320. hanges to matplotlib itself Matthew Arnison lt maffew AT cat org au gt Bug reports run d idea Prabhu Ramachandran lt prabhu_r AT users sourceforge net gt Help with X Emacs support threading patches ideas Norbert Tretkowski lt tretkowski AT inittab de gt help with Debian packaging and distribution George Sakkis lt gsakkis AT eden rutgers edu gt New matcher for tab completing named arguments of user defined functions J rgen Stenarson lt jorgen stenarson AT bostream nu gt Wildcard support implementation for searching namespaces Vivian De Smedt lt vivian AT vdesmedt com gt Debugger enhancements so that when pdb is activated from within Python coloring tab completion and other features continue to work seamlessly Scott Tsai lt scottt958 AT yahoo com tw gt Support for automatic editor invocation on syntax errors see http www scipy net roundup ipython issue36 Alexander Belchenko lt bialix AT ukr net gt Improvements for win32 paging system Will Maier lt willmaier AT ml1 net gt Official OpenBSD port Ondrej Certik lt ondrej AT certik cz gt Set up the Python docs to use the new Sphinx system used by Python Matplotlib and many more projects Stefan van der Walt lt stefan AT sun ac za gt Design and prototype of the Traits based config system History 9 2 1 Origins Python was starting in 2001 by Fernando Perez while he was a graduate student at the University of Col orado Boulder Py
321. has a DEALER socket that acts as a virtual keyboard for the kernel while this communication is happening illustrated in the figure by the black outline around the central keyboard In practice frontends may display such kernel requests using a special input widget or otherwise indicating that the user is to type input for the kernel instead of normal commands in the frontend 2 Shell this single ROUTER socket allows multiple incoming connections from frontends and this is the socket where requests for code execution object information prompts etc are made to the kernel 384 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack by any frontend The communication on this socket is a sequence of request reply actions from each frontend and the kernel 3 IOPub this socket is the broadcast channel where the kernel publishes all side effects stdout stderr etc as well as the requests coming from any client over the shell socket and its own requests on the stdin socket There are a number of actions in Python which generate side effects print writes to sys stdout errors generate tracebacks etc Additionally in a multi client scenario we want all frontends to be able to know what each other has sent to the kernel this can be useful in collaborative scenarios for example This socket allows both side effects and the information about communications taking place with one c
322. havior of the parallel magic commands See also See the documentation of the Parallel Magics 5 3 5 Moving Python objects around In addition to calling functions and executing code on engines you can transfer Python objects to and from your Python session and the engines In I Python these operations are called push sending an object to the engines and pull getting an object from the engines Basic push and pull Here are some examples of how you use push and pull In 38 dview push dict a 1 03234 b 3453 Out 38 None None None None In 39 dview pull a Out 39 1 03234 1 03234 1 03234 1 03234 In 40 dview pull b targets 0 Out 40 3453 In 41 dview pull a b Out 41 1 03234 3453 1 03234 3453 1 03234 3453 In 42 dview push dict c speed Out 42 None None None None 1 03234 3453 In non blocking mode push and pull also return AsyncResult objects In 48 ar dview pull a block False In 49 ar get Out 49 1 03234 1 03234 1 03234 1 03234 284 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack Dictionary interface Since a Python namespace is just a dict Direct View objects provide dictionary style access by key and methods such as get and update for convenienc
323. he Sconnect_info magic to display the information necessary to connect another frontend to the kernel In 2 connect_info stdin_port 50255 Wipe 127 20 Oe Ss hb port 50256 4 5 A Qt Console for IPython 239 IPython Documentation Release 1 2 1 An Afternoon Hack key 70be6f0f 1564 4218 8cda 31be40a4d6aa shell_port 50253 iopub_port 50254 Paste the above JSON into a file and connect with gt ipython lt app gt existing lt file gt or if you are local you can connect with just gt ipython lt app gt existing kernel 12345 json or even just gt ipython lt app gt existing if this is the most recent IPython session you have started Otherwise you can find a connection file by name and optionally profile with IPython lib kernel find_connection_file gt python c from IPython lib kernel import find_connection_file print find_connection_file kernel 12345 json home you ipython profile_default security kernel 12345 json And if you are using a particular Python profile gt python c from IPython lib kernel import find_connection_file print find_connection_file kernel 12345 json profile foo home you ipython profile_foo security kernel 12345 json You can even launch a standalone kernel and connect and disconnect Qt Consoles from various machines This lets you keep the same running Python session on your wo
324. he end of the line behavior different from default Python Ampersands can t be typed sometimes in notebook cells Add expand collapse support for long output elements like stdout and tracebacks error when starting ipython Ability to run notebook file from commandline Qt Console output spacing Test failures with Python 3 2 and PYTHONWARNINGS d about inline Starting Python3 on the terminal easy_install ipython fails on py3 2 win32 70 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e 2075 Local Mathjax install causes iptest3 error under python3 e 2057 setup fails for python3 with LANG C e 2070 shebang on Windows e 2054 sys_info missing git hash in sdists e 2059 duplicate and modified files in documentation e 2056 except shadows builtin osm py 687 e 2058 hyphen used as minus sign in manpages 2 4 0 13 Series 2 4 1 Release 0 13 Python 0 13 contains several major new features as well as a large amount of bug and regression fixes The previous version 0 12 was released on December 19 2011 and in this development cycle we had e 6 months of work e 373 pull requests merged e 742 issues closed non pull requests e contributions from 62 authors e 1760 commits e a diff of 114226 lines The amount of work included in this release is so large that we can only cover here the main highlights please see our detailed release statistics for links to ev
325. he fix im thinking of 4 Do some edits and commit them as you go hack hack hack Tell git about any new files you ve made git add somewhere tests test_my_bug py commit work in progress as you go git commit am BF added tests for Funny bug hack hack hack git commit am BF added fix for Funny bug 7 1 Working with IPython source code 373 IPython Documentation Release 1 2 1 An Afternoon Hack Note the am options to commit The m flag just signals that you re going to type a message on the command line The a flag you can just take on faith or see why the a flag 5 When you have finished check you have committed all your changes git status 6 Finally make your commits into patches You want all the commits since you branched from the master branch git format patch M C master You will now have several files named for the commits 0001 BF added tests for Funny bug patch 0002 BF added fix for Funny bug patch Send these files to the ipython mailing list When you are done to switch back to the main copy of the code just return to the master branch git checkout master Moving from patching to development If you find you have done some patches and you have one or more feature branches you will probably want to switch to development mode You can do this with the repository you ha
326. he same kernel The kernel now provides the ability to retrieve data from the user s namespace after the execution of the main code thanks to two fields in the execute_request message e user_variables If only variables from the user s namespace are needed a list of variable names can be passed and a dict with these names as keys and their repr as values will be returned e user_expressions For more complex expressions that require function evaluations a dict can be provided with string keys and arbitrary python expressions as values The return message will contain also a dict with the same keys and the repr of the evaluated expressions as value With this information frontends can display any status information they wish in the form that best suits each frontend a status line a popup inline for a terminal etc Note In order to obtain the current execution counter for the purposes of displaying input prompts fron tends simply make an execution request with an empty code string and silent True Execution semantics When the silent flag is false the execution of use code consists of the following phases in silent mode only the code field is executed 1 Run the pre_runcode_hook 2 Execute the code field see below for details 3 If 2 succeeds compute user_variables and user_expressions are computed This en sures that any error in the latter don t harm the main code execution 4 Call any method
327. help system gt Details about object use object for gt A brief reference about the graphical user run recarr_simple py Seismic stations in the Himalaya 4 38 E 85 51 E 86 64 E 87 77 E 88 90 xX i a Type copyright credits or license for more information 0 11 alpha1 git An enhanced Interactive Python gt Introduction and overview of IPython s features extra details interface from scipy import special as sp x linspace 20 100 for n in range 4 y sp jn n x plot x y label r J_ s x n axhline color green label _nolegend_ grid legend A xlabel x title r Bessel functions J_n x lt matplotlib text Text object at x7fcddc1795de gt Bessel functions J x 156 apter 2 What s new in IPython 20 IPython Documentation Release 1 2 1 An Afternoon Hack High level parallel computing with ZeroMQ We have completely rewritten the Twisted based code for high level parallel computing to work atop our new ZeroMQ architecture While we realize this will break compatibility for a number of users we hope to make the transition as easy as possible with our docs and we are convinced the change is worth it ZeroMQ provides us with much tighter control over memory higher performance and its communications are impervious to the Python Global Interpreter Lock because they take place in a system level C thread The impa
328. hen starting the server See the Notebook docs for technical details Two process terminal console Based on the same architecture as the notebook and the Qt console we also have now a terminal based console that can connect to an external Python kernel the same kernels used by the Qt console or the notebook in fact While this client behaves almost identically to the usual Python terminal application this capability can be very useful to attach an interactive console to an existing kernel that was started externally It lets you use the interactive Sdebug facilities in a notebook for example the web browser can t interact directly with the debugger or debug a third party code where you may have embedded an Python kernel This is also something that we have wanted for a long time and which is a culmination as a team effort of the work started last year during the 2010 Google Summer of Code project Tabbed QtConsole The QtConsole now supports starting multiple kernels in tabs and has a menubar so it looks and behaves more like a real application Keyboard enthusiasts can disable the menubar with ctrl shift M PR 887 Full Python 3 compatibility Python can now be installed from a single codebase on Python 2 and Python 3 The installation process for Python 3 automatically runs 2to3 The same default profile is now used for Python 2 and 3 the previous version had a separate python3 profile Standalone Kernel
329. his means that almost no time is spent in handling the request assuming an Emacs process is already running For this to work you need to set your EDITOR environment variable to emacsclient The code below supplied by Francois Pinard can then be used in your emacs file to enable the server defvar server buffer clients when and fboundp server start string equal getenv TERM xterm server start defun fp kill server with buffer routine and server buffer clients server done add hook kill buffer hook fp kill server with buffer routine Thanks to the work of Alexander Schmolck and Prabhu Ramachandran currently X Emacs and Python get along very well in other ways Note You will need to use a recent enough version of python mode el along with the file ipython el You can check that the version you have of python mode el is new enough by ei ther looking at the revision number in the file itself or asking for it in X Emacs via M x py version Versions 4 68 and newer contain the necessary fixes for proper Python support The file ipython e1 is included with the Python distribution in the directory docs emacs Once you put these files in your Emacs path all you need in your emacs file is require ipython This should give you full support for executing code snippets via Python opening Python as your Python shell via C c etc You can cust
330. hods can be called on any number of sequences but they can also take the block and bound keyword arguments just like apply but only as keywords dview map sequences block None e iter map_async reduce 5 13 10 Decorators and RemoteFunctions Note TODO write this section parallel remote RemoteFunction ParallelFunction 5 13 Details of Parallel Computing with IPython 341 IPython Documentation Release 1 2 1 An Afternoon Hack 5 13 11 Dependencies Note TODO write this section depend require Dependency 5 14 Transitioning from IPython kernel to IPython parallel We have rewritten our parallel computing tools to use OMQ and Tornado The redesign has resulted in dra matically improved performance as well as we think an improved interface for executing code remotely This doc is to help users of Python kernel transition their codes to the new code 5 14 1 Processes The process model for the new parallel code is very similar to that of Python kernel There is still a Controller Engines and Clients However the the Controller is now split into multiple processes and can even be split across multiple machines There does remain a single ipcontroller script for starting all of the controller processes Note TODO fill this out after config system is updated See also Detailed Parallel Process doc for configuring and launching Python processes
331. hods in firefox Windows home directory expansion test fails Fix home path expansion test in Windows Windows core test error filename quoting Fix Windows pathname issue in odd encoding test call to nt assert_true path _writable_dir home returns false in test_path py don t check writability in test for get_home_dir when HOME is undefined Test failures and docs don t build on Mac OS X Lion frontend test tweaks Notebook server can t start cluster with hyphen containing profile name relax profile regex in notebook brew install pyqt Strange output artifacts in footer of notebook Mono cursor offset Clarify generic message spec vs Python message API in docs Don t crash when starting notebook server if runnable browser not found notebook Print a warning but do not abort if no webbrowser can be found pip install virtualenv Wrong case of rmagic in docs 104 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 2002 2000 1999 1635 1984 1981 1994 1949 1985 1980 1951 1969 1974 1964 1960 1991 1982 1988 1979 1989 1966 1987 1970 1983 1978 1553 1825 1915 1934 1746 1790 1767 Refactor magic into a lsmagic_docs API function kernel js consistency with generic Python message format Yomagic help display line and cell magics in alphabetical order test_prun_quotes fails on Windows Ca
332. hon Set up your fork and you have configured git by following Configure git 7 1 Working with IPython source code 377 IPython Documentation Release 1 2 1 An Afternoon Hack Workflow summary e Keep your master branch clean of edits that have not been merged to the main ipython development repo Your master then will follow the main ipython repository e Start a new feature branch for each set of edits that you do e If you can avoid it try not to merge other branches into your feature branch while you are working e Ask for review This way of working really helps to keep work well organized and in keeping history as clear as possible See for example linux git workflow Making a new feature branch git branch my new feature git checkout my new feature Generally you will want to keep this also on your public github fork of ipython To do this you git push this new branch up to your github repo Generally if you followed the instructions in these pages and by default git will have a link to your github repo called origin You push up to your own repo on github with git push origin my new feature From now on git will know that my new feature is related to the my new feature branch in the github repo The editing workflow Overview hack hack git add my_new_file git commit am NF some message git push In more detail 1 Make some changes
333. hon notebook pylab inline import_all False Mathjax Utils race condition 2 3 Issues closed in the 1 0 development cycle 51 IPython Documentation Release 1 2 1 An Afternoon Hack 1540 2702 3672 2411 481 2664 3436 2630 3154 3579 3604 3668 3245 3639 Comment uncomment selection in notebook frozen setup permission denied for default ipython_dir allow_none on Number like traits add CONTRIBUTING md Python terminal issue with Qt4Agg on XP SP3 How to preserve user variables from import clashing enable_pylab import_all False still imports np lib pylabtools figsize NameError when using Qt4Agg backend and pylab magic Notebook no event triggered when a Cell is created Nbconvert SVG are not transformed to PDF anymore MathJax rendering problem in latex cell AttributeError BlockingKernelClient object has no attribute started_channels SyntaxError encoding declaration in Unicode string pylab inline in IPYTHON notebook throws RuntimeError Cannot activate multiple GUI eventloops 3663 3661 3597 3053 3655 3620 3646 3004 10 8 3638 3634 1307 33106 3494 3427 3625 3149 3617 frontend deprecation warnings run m not behaving like python m re do PR 3531 allow markdown in Header cell Markdown in header cells is not rendered Python finding its way into pasted strings uncaught errors in HTML output
334. hout readline but will lack many features that users depend on such as tab completion and command line editing If you install Python with setuptools e g with easy_install then the appropriate readline for your platform will be installed See below for details of how to make sure you have a working readline 3 3 1 Installation using easy_install or pip If you have setuptools or pip the easiest way of getting Python is to simply use easy_instal1 or pip S pip install ipython That s it Note Many prefer pip to easy_instal11 but it ignores eggs binary Python packages This mainly affects pyzmq and readline which are compiled packages and provide binary eggs If you use pip to install these packages it will always compile from source which may not succeed 196 Chapter 3 Installation IPython Documentation Release 1 2 1 An Afternoon Hack 3 3 2 Installation from source If you don t want to use easy_instal1l1 or don t have it installed just grab the latest stable build of Python from here Then do the following S tar xzf ipython tar gz S cd ipython S python setup py install If you are installing to a location like usr local1 that requires higher permissions you may need to run the last command with sudo 3 3 0 Windows As mentioned above on Windows Python requires setuptool1s and it also requires the PyReadline library to properly support coloring and keyboard ma
335. hs 6 3 2 Classes that can be configured The following classes can also be configured in the configuration file for ipython e Interactiveshell e PrefilterManager e AliasManager 6 3 Configuring the ipython command line application 359 IPython Documentation Release 1 2 1 An Afternoon Hack To see which attributes of these classes are configurable please see the source code for these classes the class docstrings or the sample configuration file IPython config default ipython_config 6 3 3 Example For those who want to get a quick start here is a sample ipython_config py that sets some of the common configuration attributes sample ipython_config py c get_config c TerminallIPythonApp display_banner True InteractiveShellApp log_level 20 c InteractiveShellApp extensions myextension Q c InteractiveShellApp exec_lines import numpy import scipy c InteractiveShellApp exec_files mycode py fancy ipy c InteractiveShell autoindent True c InteractiveShell colors LightBG c InteractiveShell confirm_exit False c InteractiveShell deep_reload Tru c InteractiveShell editor nano c InteractiveShell xmode Context c PromptManager in_template In c PromptManager in2_template ADi c PromptManager out_template Out c PromptManager justify True c PrefilterManager multi_line_sp
336. ic publishing of HTML notebook use argparse to parse aliases amp flags HistoryManager get_session_info Fix columnize bug where tab completion with very long filenames would crash Qt console add ssh tunnel support to qtconsole Add SSH tunneling to engines Allow pickling objects defined interactively My fix rpmlint don t special case for py3k numpy make config loading debug messages more explicit make calltips configurable in qtconsole parallel tests amp extra readline escapes BF allow nose with doctest setting in environment Protect ipkernel from bad messages Prevent ipython py launcher from being imported Prevent ipython py from being imported by accident check for writable dirs not just existence in utils path Sessionwork add ipython kernel for starting just a kernel Qt Console history search Generate package list automatically in find_packages 1658 don t crash on bad config files Regular issues 258 e 1177 UnicodeDecodeError in py3compat from xlrd 1094 Tooltip doesn t show constructor docstrings 1170 double pylab greeting with c InteractiveShellApp pylab tk in zmqconsole 1166 E mail cpaste broken 1164 Python qtconsole 0 12 can t export to html with external png 144 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e 1103 loadpy should cut out encoding declaration e 1156 Notebooks downloaded as Python files requ
337. ielsen lt jenshnielsen at gmail com gt Johann Cohen Tanugi lt johann cohentanugi at gmail com gt J rgen Stenarson lt jorgen stenarson at bostream nu gt Justin Riley lt justin t riley at gmail com gt Kiorky Laurent Dufrechou lt laurent dufrechou at gmail com gt Luis Pedro Coelho lt Ipc at cmu edu gt 164 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e Mani chandra lt mchandra at titk ac in gt e Mark E Smith e Mark Voorhies lt mark voorhies at ucsf edu gt e Martin Spacek lt git at mspacek mm st gt Michael Droettboom lt mdroe at stsci edu gt e MinRK lt benjaminrk at gmail com gt muzuiget lt muzuiget at gmail com gt Nick Tarleton lt nick at quixey com gt Nicolas Rougier lt Nicolas rougier at inria fr gt Omar Andres Zapata Mesa lt andresete chaos at gmail com gt Paul Ivanov lt pivanov314 at gmail com gt Pauli Virtanen lt pauli virtanen at iki fi gt Prabhu Ramachandran Ramana lt sramana9 at gmail com gt Robert Kern lt robert kern at gmail com gt Sathesh Chandra lt satheshchandra88 at gmail com gt Satrajit Ghosh lt satra at mit edu gt Sebastian Busch Skipper Seabold lt jsseabold at gmail com gt Stefan van der Walt lt bzr at mentat za net gt Stephan Peijnik lt debian at sp or at gt Steven Bethard Thomas Kluyver lt takowl at gmail com gt Thomas Spura lt tomspur at fedoraproject org gt T
338. if file already exists add save f pyreadline version dependency not correctly checked add version checking to pyreadline import test Octave magics 106 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 1759 1931 1646 1933 1916 1932 1928 1923 1926 1929 1409 1922 1598 1893 1890 1921 1919 1918 1914 1917 1428 1913 1911 1549 1908 1909 1907 1904 1906 1894 1799 1845 github merge PR s just by number s Win py3fixes Meaning of restart parameter in client shutdown unclear oinspect find_file Additional safety if file cannot be found paste doesn t work on py3 Fix adding functions to CommandChainDispatcher with equal priority on Py 3 Select NoDB by default Add IPython syntax support to the timeit magic in line and cell mode Make completer recognize escaped quotes in strings Ipython qtconsole 0 12 1 hangs with Python 2 7 3 Windows 7 64 bit qtconsole forward delete bring completion into current line py3k compatibility for setupegg py document that sync_imports can t handle import foo as bar Update Parallel Magics and Exception Display Docstrings for magics that use magic_arguments are rendered wrong magic_arguments dedent but otherwise preserve indentation Use oinspect in CodeMagics _find_edit_target don t warn in iptest if deathrow quarantine are missing pdef failin
339. ifficult bug which is actually a problem in Python itself meant long running sessions would inevitably grow in memory size often with catastrophic conse quences if users had large objects in their scripts Now using run repeatedly should not cause any memory leaks Special thanks to John Hunter and Sameer D Costa for their help with this bug Fix 295371 bug in history Improved support for py2exe Fix 270856 Python hangs with PyGTK Fix 270998 A magic with no docstring breaks the omagic magic fix 271684 c startup commands screw up raw vs native history Numerous bugs on Windows with the new ipcluster have been fixed 2 10 0 10 series 187 IPython Documentation Release 1 2 1 An Afternoon Hack The ipengine and ipcontroller scripts now handle missing furl files more gracefully by giving better error messages rehashx Aliases no longer contain dots python3 0 binary will create alias python30 Fixes 259716 commands with dots in them don t work Yocpaste Yocpaste r repeats the last pasted block The block is assigned to pasted_block even if code raises exception Bug 274067 The code in get_home_dir is broken for py2exe was fixed Many other small bug fixes not listed here by number see the bzr log for more info Backwards incompatible changes ipykit and related files were unmaintained and have been removed The IPython genutils doctest_reload does not actually call relo
340. ig stages in Application Encoding problems otool should not be unconditionally called on osx Get profile and config file inheritance working qtconsole pure TypeError string indices must be integers not str qtconsole pure KeyError ismagic qtconsole pure TypeError string indices must be integers not str resubmitted tasks sometimes stuck as pending JSON serialization problems with ObjectId type MongoDB Channels should be named for their function not their socket type ipcluster termination on os x qtconsole indentation problem printing numpy arrays AssertionError Missing message part in ipython qtconsole pure ZMQEtror Interrupted system call when saving inline figure edit magic fails in qtconsole Don t show info from pyd files QFont setPointSize Point size lt 0 0 must be greater than 0 UnicodeEncodeError in qt svg save_svg embed doesn t load default config Using Python with RubyPython leads to problems with IPython parallel client client Client __init 401 168 Race condition when running lbview apply fast multiple times in loop Scrub Launchpad links from code docs 2 9 Issues closed in the 0 11 development cycle 175 IPython Documentation Release 1 2 1 An Afternoon Hack 141 garbage collection problem revisited 59 test_magic test_obj_del fails on win32 457 297 110 473 466 468 465 467 463 446 438
341. ilable as get_ipython banner in case you want it 222 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack InteractiveShellEmbed instances print the following information everytime thdy start A global startup banner A call specific header string which you can use to indicate where in the execution flow the shell is starting They also print an exit messag very time they exit Both the startup banner and th xit message default to None and can be set either at the instance constructor or at any other time with the by setting the banner and exit_msg attributes H H The shell instance can be also put in dummy mode globally or on a per call basis This gives you fine control for debugging without having to change code all over the place SH The code below illustrates all this This is how the global banner and exit_msg can be reset at any point ipshell banner Entering interpreter New Banner ipshell exit_msg Leaving interpreter New exit_msg def foo m s spam ipshell xIn foo Try Swhos or print s or mi print foo says m m def bar n s eggs ipshell x xIn bar Try Swhos or print s or m print bar says n n Some calls to the above functions which will trigger IPython print Main program calling foo eggs n foo eggs The shell can be p
342. ilable to you See these examples of png htm1 and svg xhtm1 output Note that syntax highlighting does not survive export This is a known issue and is being investigated 4 5 4 Colors and Highlighting Terminal Python has always had some coloring but never syntax highlighting There are a few simple color choices specified by the colors flag or colors magic e LightBG for light backgrounds e Linux for dark backgrounds 4 5 A Qt Console for IPython 237 IPython Documentation Release 1 2 1 An Afternoon Hack e NoColor for a simple colorless terminal The Qt widget has full support for the colors flag used in the terminal shell The Qt widget however has full syntax highlighting as you type handled by the pygments library The style argument exposes access to any style by name that can be found by pygments and there are sev eral already installed The colors argument if unspecified will be guessed based on the chosen style Similarly there are default styles associated with each colors option Screenshot of ipython qtconsole colors linux which uses the monokai theme by default Type type Base Class lt type type gt String Form lt type str gt Namespace Python builtin Docstring strCobject gt string Return a nice string representation of the object If the argument is a string the return value is the same object tor fCa b 5 q docstring i in range 1
343. imited builtin pager 6 7 4 Fine tuning your prompt IPython s prompts can be customized using a syntax similar to that of the bash shell Many of bash s escapes are supported as well as a few additional ones We list them below the prompt history count number This escape is automatically wrapped in the coloring codes for the currently active color scheme N the naked prompt history count number this is just the number itself without any coloring applied to it This lets you produce numbered prompts with your own colors D the prompt history count with the actual digits replaced by dots Used mainly in continuation prompts prompt_in2 6 7 Outdated configuration information that might still be useful 367 IPython Documentation Release 1 2 1 An Afternoon Hack w the current working directory W the basename of current working directory Xn where n 0 ldots5 The current working directory with HOME replaced by and filtered out to contain only n path elements Yn Similar to Xn but with the n 1 element included if it is this is similar to the behavior of the cn escapes in tcsh u the username of the current user if the effective UID is 0 a otherwise a h the hostname up to the first H the hostname n a newline r a carriage return v IPython version string In addition to these ANSI color escapes can be insterted into the prom
344. in the 1 0 development cycle 29 IPython Documentation Release 1 2 1 An Afternoon Hack PR 3590 PR 3581 IPython sphinxext needs an __init__ py Added the possibility to read a custom css file for tweaking the final html in full_html and reveal templates PR 3576 PR 3575 PR 3569 PR 3567 PR 3565 PR 3531 PR 3577 PR 3495 PR 3578 PR 3572 PR 3574 PR 3558 PR 3561 PR 3566 PR 3564 PR 3540 PR 3527 PR 3529 PR 3541 PR 3550 PR 3557 PR 3556 PR 3552 PR 3554 PR 3546 PR 3551 PR 3533 PR 3498 PR 3548 Added support for markdown in heading cells when they are nbconverted tweak run d message to continue execution add PYTHONSTARTUP to startup files Trigger a single event on js app initilized style min css shoudl always exist allow markdown in heading cells Simplify codemirror ipython mode Simplified regexp and suggestions for clearer regexps Use adjustbox to specify figure size in nbconvert gt latex Skip import irunner test on Windows correct static path for CM modes autoload Add IPython sphinxext mention double control C to stop notebook server fix event names Remove trivial nbconvert example allow cython cache dir to be deleted cleanup stale unused exceptions in parallel error ensure raw_input returns str in zmq shell respect image size metadata in qtconsole Fixing issue preventing the correct read of images by full_html and revea
345. ing PYTHONSTARTUP LaTeX escaping bug in nbconvert when exporting to HTML Long running notebook died with a coredump Running ipython with pypy on windows custom js not working IPython lib test failure on Windows Install Sphinx extensions notebook user needs to press ctrl c twice to stop notebook server should be put into terminal window 2413 2618 2595 2586 2714 2669 3256 3010 2044 ipython3 qtconsole fails to install ipython 0 13 has no such extra feature qtconsole documentation is incorrect for install process mac 10 8 qtconsole export history cannot store aliases ipython qtconsole print unittest messages in console instead his own window cython magic failing to work with openmp Vagrant pandas instance of iPython Notebook does not respect additional plotting arguments cython magic fail if cache dir is deleted while in session prune unused names from parallel error 2 3 Issues closed in the 1 0 development cycle 53 IPython Documentation Release 1 2 1 An Afternoon Hack 1145 3439 3476 3499 2771 2887 626 2824 2823 2891 2928 2931 2939 2972 Online help utility broken in QtConsole Markdown links no longer open in new window with change from pagedown to marked _margv for macros seems to be missing Add reveal js library version 2 4 0 inside Python Wiki Migration to GitHub ipcontroller purging some engines during connect Enable Resumi
346. ing non ascii history 797 Source introspection needs to be smarter in python 3 2 846 Autoreload extension doesn t work with Python 3 2 1360 Python notebook not starting on winXP 1407 Qtconsole segfaults on OSX when displaying some pop up function tooltips 1402 fix symlinked home issue for FreeBSD 1403 pyreadline cyclic dependency with pdb pdbpp module 1405 Only monkeypatch xunit when the tests are run using it 1404 Feature Request List Dictionary tab completion 1395 Xunit amp KnownFailure 1396 Fix for tb magic 1397 Stay or leave message not working Safari session lost 1389 pylab inline inoperant through ssh tunnelling 1386 Jsd3 1388 Add simple support for running inside a virtualenv 826 Add support for creation of parallel task when no engine is running 1391 Improve Hub Scheduler when no engines are registered 1369 load header with engine id when engine dies in TaskScheduler 1345 notebook can t save unicode as script 1353 Save notebook as script using unicode file handle 1352 Add m mod run library module as a script option 1363 Fix some minor color style config issues in the qtconsole 1371 Adds a quiet keyword to sync_imports 1390 Blank screen for notebooks on Safari 2 5 Issues closed in the 0 13 development cycle 121 IPython Documentation Release 1 2 1 An Afternoon Hack 1387 Fixing Cell menu to update cell type sel
347. ing the default values of targets blocking for the active View Output Display px also accepts a group outputs argument which adjusts how the outputs of multiple engines are presented See also AsyncResult display_outputs for the grouping options In 50 px block group outputs engine import numpy as np 1 A np random random 2 2 1 ev numpy linalg eigvals A 1 print ev 1 CV max Parallel execution on engine s all stdout 0 0 60640442 0 95919621 Out 0 73 0 9591962130899806 5 4 Parallel Magic Commands 291 IPython Documentation Release 1 2 1 An Afternoon Hack stdout 1 0 38501813 1 29430871 Out 1 73 1 2943087091452372 stdout 2 0 85925141 0 9387692 Out 2 73 0 93876920456230284 stdout 3 0 37998269 1 24218246 Out 3 73 1 2421824618493817 pxresult If you are using px in non blocking mode you won t get output You can use pxresult to display the outputs of the latest command just as is done when px is blocking In 39 dv block False In 40 px print hi Async parallel execution on engine s all In 41 spxresult stdout 0 hi stdout 1 hi stdout 2 hi stdout 3 hi pxresult simply calls AsyncResult display_outputs on the most recent request It accepts the same output grouping arguments as px so you can use it to view a result in different ways aut
348. ing values is pure Python assignment Simple flags exist for commonly used options these are always prefixed with The Python command line help has the details of all the options via ipythyon help but a simple example should clarify things the py1lab flag can be used to start in pylab mode with the qt4 backend ipython pylab qt which is equivalent to using the fully qualified form ipython TerminallIPythonApp pylab qt The long form options can be listed via ipython help all 2 8 0 11 Series 157 IPython Documentation Release 1 2 1 An Afternoon Hack ZeroMQ architecture There is a new GUI framework for Python based on a client server model in which multiple clients can communicate with one Python kernel using the ZeroMQ messaging framework There is already a Qt console client which can be started by calling ipython qtconsole The protocol is documented The parallel computing framework has also been rewritten using ZMQ The protocol is described here and the code is in the new IPython parallel module Python 3 support A Python 3 version of Python has been prepared For the time being this is maintained separately and updated from the main codebase Its code can be found here The parallel computing components are not perfect on Python3 but most functionality appears to be working As this work is evolving quickly the best place to find updated information
349. install an Python extension from a URL E g S install_ext https bitbucket org birkenfeld ipython physics raw default phys The 3loadpy magic is no longer restricted to Python files and has been renamed load The old name remains as an alias New command line arguments will help external programs find Python folders ipython locate finds the user s Python directory and ipython locate profile foo finds the folder for the foo profile if it exists The IPYTHON_DIR environment variable introduced in the Great Reorganization of 0 11 and ex isting only in versions 0 11 0 13 has been deprecated As described in PR 1167 the complexity and confusion of migrating to this variable is not worth the aesthetic improvement Please use the historical TPYTHONDIR environment variable instead The default value of interactivity passed from run_cell to run_ast_nodes is now config urable New alias_magic function to conveniently create aliases of existing magics if you prefer to have shorter names for personal use We ship unminified versions of the JavaScript libraries we use to better comply with Debian s pack aging policies Simplify the information presented by 0b3 ob j to eliminate a few redundant fields when pos sible PR 2038 Improved continuous integration for Python We now have automated test runs on Shining Panda and Travis ClI as well as Tox support The vim ipython functionality externally develo
350. ion object you simply set attributes on it All you have to know is e The name of each attribute e The type of each attribute The answers to these two questions are provided by the various Configurable subclasses that an appli cation uses Let s look at how this would work for a simple configurable subclass Sample configurable from IPython config configurable import Configurable from IPython utils traitlets import Int Float Unicode Bool class MyClass Configurable name Unicode u defaultname config True ranking Int 0 config True 348 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack value Float 99 0 The rest of the class implementation would go here In this example we see that MyClass has three attributes two of whom name ranking can be con figured All of the attributes are given types and default values If a MyClass is instantiated but not configured these default values will be used But let s see how to configure this class in a configuration file Sample config file c get_config c MyClass name coolname c MyClass ranking 10 After this configuration file is loaded the values set in it will override the class defaults anytime aMyClass is created Furthermore these attributes will be type checked and validated anytime they are set This type checking is handled by the TPython utils
351. ion of the APIs thus matching what most applications would expect It will fall back of PySide if PyQt4 is unavailable If specified Python will respect the environment variable QT_API used by ETS ETS 4 0 also works with both PyQt4 and PySide but it requires PyQt4 to use its v2 API So if OT_API pyside PySide will be used and if OT_API pyqt then PyQt4 will be used with the v2 API for QString and QVariant so ETS codes like MayaVi will also work with IPython If you launch Python in matplotlib mode with ipython matplotlib qt then Python will ask matplotlib which Qt library to use only if QT_API is not set via the backend qt4 rcParam If matplotlib is version 1 0 1 or older then Python will always use PyQt4 without setting the v2 APIs since neither v2 PyQt nor PySide work Warning Note that this means for ETS 4 to work with PyQt4 QT_API must be set to work with IPython s qt integration because otherwise PyQt4 will be loaded in an incompatible mode It also means that you must not have OQT_APT set if you want to use gui qt with code that requires PyQt4 API v1 4 3 8 Plotting with matplotlib matplotlib provides high quality 2D and 3D plotting for Python matplotlib can produce plots on screen using a variety of GUI toolkits including Tk PyGTK PyQt4 and wxPython It also provides a number of commands useful for scientific computing all with a syntax compatible with that of the popular Matlab program To
352. ire a header stating the encoding e 1157 Ctrl C not working when GUI pylab integration is active e 1154 We should be less aggressive in de registering post execution functions e 1134 select all kill leaves qtconsole in unusable state e 1148 A lot of testerrors 803 Make doctests work with Python 3 1119 Start menu shortcuts not created in Python 3 1136 The embedding machinery ignores user_ns 607 Use the new Python logo font in the notebook header 755 qtconsole ipython widget s execute_file fails if filename contains spaces or quotes 1115 shlex_split should return unicode 1109 timeit with string ending in space gives ValueError No closing quotation 1142 Install problems 700 Some SVG images render incorrectly in htmlnotebook 1117 quitQ doesn t work in terminal 1111 ls broken after merge of 1089 1104 Prompt spacing weird 1124 Seg Fault 11 when calling PySide using run command 1088 QtConsole can t copy from pager 568 Test error and failure in Python core on windows 1112 testfailure in Python frontend on windows 1102 magic in IPythonDemo fails when not located at top of demo file 629 r and b in qtconsole don t behave as expected 1080 Notebook tab completion should close on 973 Qt Console close dialog and on top Qt Console 1087 QtConsole xhtml Svg export broken 1067 Parallel test suite hangs on Python 3 1018 Local mathjax breaks install
353. is see examples parallel customresult py 5 6 The AsyncResult object 305 IPython Documentation Release 1 2 1 An Afternoon Hack 5 7 Using MPI with IPython Often a parallel algorithm will require moving data between the engines One way of accomplishing this is by doing a pull and then a push using the multiengine client However this will be slow as all the data has to go through the controller to the client and then back through the controller to its final destination A much better way of moving data between engines is to use a message passing library such as the Mes sage Passing Interface MPI MPI IPython s parallel computing architecture has been designed from the ground up to integrate with MPI This document describes how to use MPI with IPython 5 7 1 Additional installation requirements If you want to use MPI with IPython you will need to install e A standard MPI implementation such as OpenMPI OpenMPI or MPICH e The mpi4py mpi4py package Note The mpi4py package is not a strict requirement However you need to have some way of calling MPI from Python You also need some way of making sure that MPT_Tnit is called when the Python engines start up There are a number of ways of doing this and a good number of associated subtleties We highly recommend just using mpi4py as it takes care of most of these problems If you want to do something different let us know and we can help you get started
354. is cycle from August 1 to December 28 2011 we closed a total of 515 issues 257 pull requests and 258 regular issues this is the full list generated with the script tools github_stats py Pull requests 257 1174 1178 1096 1176 1173 1168 1165 1161 1158 1160 Remove install_default_config and install_profiles Correct string type casting in pinfo Show class init and call tooltips in notebook Modifications to profile list don t load gui pylab in console frontend Add script flag as shorthand for notebook save_script option encode image_tag as utf8 in x html export Allow loadpy to load remote URLs that don t end in py Add coding header when notebook exported to py file don t ignore ctrl C during gui qt 136 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e 1159 e 1155 s 1153 e 1152 e 1079 e 1151 e 1120 e 1150 e 1149 e 1147 e 1146 e 1128 e 1140 e 1132 e 1125 e 1135 s 1122 e 1137 e 1130 e 1116 e 1073 e 1143 e 1138 e 1121 e 1118 e 1105 e 1126 e 1091 e 1114 e 1113 Add encoding header to Python files downloaded from notebooks minor post execute fixes 1154 Pager tearing bug Add support for displaying maptlotlib axes directly Login out button cleanups allow access to user_ns in prompt_manager updated vim ipython pending BUG Scrolling pager in vsplit on Mac OSX tears 1148 win
355. is design e Do we use this message type for pyout displayhook Probably not because the displayhook also has to handle the Out prompt display On the other hand we could put that information into the metadata secion Message type display_data content Who create the data source str The data dict contains key value pairs where the kids are MIME types and the values are the raw data of the representation in that format data gt dret Any metadata that describes the data metadata dict The metadata contains any metadata that describes the output Global keys are assumed to apply to the output as a whole The metadata dict can also contain mime type keys which will be sub dictionaries which are interpreted as applying only to output of that type Third parties should put any data they write into a single dict with a reasonably unique name to avoid conflicts The only metadata keys currently defined in Python are the width and height of images metadata image png width 640 height 480 Raw Data Publication display_data lets you publish representations of data such as images and html This data_pub message lets you publish actual raw data sent via message buffers data_pub messages are constructed via the Python 1lib datapub publish_data function from IPython kernel zmq datapub import publish_data ns dict x my_array publish_data
356. is no way to know what tasks are on which engine until they finish This makes recovery from engine shutdown very difficult Note TODO performance comparisons 5 5 7 More details The LoadBalancedView has many more powerful features that allow quite a bit of flexibility in how tasks are defined and run The next places to look are in the following classes e LoadBalancedView e AsyncResult e apply e dependency The following is an overview of how to use these classes together 1 Create a Client and LoadBalancedView 2 Define some functions to be run as tasks 3 Submit your tasks to using the apply method of your LoadBalancedView instance 4 Use Client get_result to get the results of the tasks or use the AsyncResult get method of the results to wait for and then receive the results See also A demo of DAG Dependencies with NetworkX and Python 5 6 The AsyncResult object In non blocking mode apply submits the command to be executed and then returns a AsyncResult object immediately The AsyncResult object gives you a way of getting a result at a later time through its get method but it also collects metadata on execution 5 6 1 Beyond multiprocessing s AsyncResult Note The AsyncResult object provides a superset of the interface in 302 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack multiprocessing pool AsyncResul
357. it will pickle it There is a short list of objects that are not pickled buffers str bytes objects and numpy arrays These are handled specially by Python in order to prevent the copying of data Sending bytes or numpy arrays will result in exactly zero in memory copies of your data unless the data is very small If you have an object that provides a Python buffer interface then you can always send that buffer without copying and reconstruct the object on the other side in your own code It is possible that the object reconstruction will become extensible so you can add your own non copying types but this does not yet exist Closures Just about anything in Python is pickleable The one notable exception is objects generally functions with closures Closures can be a complicated topic but the basic principal is that functions that refer to variables in their parent scope have closures An example of a function that uses a closure def f a def inner inner will have a closure return a return inner fl 1 f2 2 f1 returns 1 2 returns 2 f1 and f2 will have closures referring to the scope in which inner was defined because they use the variable a As a result you would not be able to send f1 or 2 with IPython Note that you would be able to send f This is only true for interactively defined functions as are often used in decorators and only when there are variables used inside the inn
358. iter ipy dev meeting semantic versionify version number for non releases Documentation txt to rst cleanup terminal console iopub handling Fix for 3719 re raise KeyboardInterrupt in raw_input Organizing reveal s templates Use link 2 when possible in nbconvert skip tests that require pandoc add Importing Notebooks example nbconvert Add cwd to sys path fix raw_input in qtconsole document the wire protocol convert Python syntax to Python syntax in nbconvert python template Closes 3788 26 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 3794 PR 3746 PR 3781 PR 3779 PR 3743 PR 3732 PR 3761 PR 3758 PR 3769 PR 3771 PR 3741 PR 3753 PR 3766 PR 3736 PR 3759 PR 3760 PR 3750 PR 3748 PR 3734 PR 3733 PR 3722 PR 3704 Change logo link to ipython org Raise a named exception when pandoc is missing comply with the message spec in the notebook remove bad if logged_in preventing new notebook without login remove notebook read only view add delay to autosave in beforeunload Added rm_math_space to markdown cells in the basichtml tpl to be rendered ok by math jax after the nbconvertion nbconvert Filter names cleanup Add configurability to tabcompletion timeout Update px pylab test to match new output of pylab better message when notebook format is not supported document Ctrl C not working in ipython k
359. ith a spaces or tabs the same whitespace will b removed from each following line until it is reset UL space_re re compile r t line while True line yield line if line is None continue m space_re match line if m space m group 0 while line is not None if line startswith space line line len space line yield line else No leading spaces wait for reset while line is not None 364 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack line yield line leading_indent look_in_string True Token based transformers There is an experimental framework that takes care of tokenizing and untokenizing lines of code De fine a function that accepts a list of tokens and returns an iterable of output tokens and decorate it with TokenInputTransformer wrap These should only be used in python_line_transforms 6 6 2 AST transformations After the code has been parsed as Python syntax you can use Python s powerful Abstract Syntax Tree tools to modify it Subclass ast NodeTransformer and add an instance to shell ast_transformers This example wraps integer literals in an Integer class which is useful for mathematical frameworks that want to handle e g 1 3 as a precise fraction class IntegerWrapper ast NodeTransformer Wraps all integers in a call to Integer def visit
360. ithub merge PR s just by number s Win py3fixes oinspect find_file Additional safety if file cannot be found Fix adding functions to CommandChainDispatcher with equal priority on Py 3 Select NoDB by default Add IPython syntax support to the timeit magic in line and cell mode Make completer recognize escaped quotes in strings Update Parallel Magics and Exception Display magic_arguments dedent but otherwise preserve indentation Use oinspect in CodeMagics _find_edit_target don t warn in iptest if deathrow quarantine are missing Fix for pdef on Python 3 Fix for 1428 temporarily skip autoreload tests Fix for 1908 use os path normcase for safe filename comparisons py3compat fixes for script and tests ofind finds non unique cell magics Fixes to inspection machinery for magics Workaround fix for gh 1632 minimal revert of gh 1424 Cython libs add ScriptMagics to class list for generated config minimize manpages use glob for bad exclusion warning 2 5 Issues closed in the 0 13 development cycle 93 IPython Documentation Release 1 2 1 An Afternoon Hack PR 1855 PR 1870 PR 1861 PR 1867 PR 1889 PR 1886 PR 1895 PR 1762 PR 1883 PR 1875 PR 1851 PR 1892 PR 1864 PR 1829 PR 1832 PR 1674 PR 1882 PR 1879 PR 1841 PR 1850 PR 1663 PR 1815 PR 1819 PR 1839 PR 1782 PR 1862 PR 1857 PR 1856 PR 1852 PR 1780 PR 1847 PR 1836 7 script and
361. ive_binomial random_sample Fig 2 2 The new improved Qt console s ncurses like completer allows to easily navigate thought long list of completions In the notebook completions are now sourced both from object introspection and analysis of surrounding code so limited completions can be offered for variables defined in the current cell or while the kernel is busy PR 1711 We have implemented a new configurable flag to control tab completion on modules that provide the __all___attribute IPCompleter limit_to__all__ Boolean This instructs the completer to honor __al1l__ for the completion Specifically when completing on object lt tab gt if True only those names in obj ___al11___ will be included When False default the_all_ attribute is ignored PR 1529 Improvements to the Qt console The Qt console continues to receive improvements and refinements despite the fact that it is by now a fairly mature and robust component Lots of small polish has gone into it here are a few highlights e A number of changes were made to the underlying code for easier integration into other projects such as Spyder PR 2007 PR 2024 2 4 0 13 Series 83 IPython Documentation Release 1 2 1 An Afternoon Hack e Improved menus with a new Magic menu that is organized by magic groups this was made possible by the reorganization of the magic system internals PR 1782 e Allow for restarting kernels without clearing
362. k e The serialized header dict e The serialized parent header dict e The serialized metadata dict e The serialized content dict In Python this is implemented via once digester HMAC key digestmod hashlib sha256 for each message d digester copy for serialized_dict in header parent metadata content d update serialized_dict Signature d hexdigest After the signature is the actual message always in four frames of bytes The four dictionaries that compose a message are serialized separately in the order of header parent header metadata and content These can be serialized by any function that turns a dict into bytes The default and most common serialization is JSON but msgpack and pickle are common alternatives After the serialized dicts are zero to many raw data buffers which can be used by message types that support binary data mainly apply and data_pub 7 2 4 Python functional API As messages are dicts they map naturally toa func kw call form We should develop at a few key points functional forms of all the requests that take arguments in this manner and automatically construct the necessary dict for sending In addition the Python implementation of the message specification extends messages upon deserialization to the following form for convenience header dict The msg s unique identifier and type are always stored in the header but the Python
363. k has been renamed IPython kernel client StringTask to make way for new task types The keyword argument style has been renamed dist in scatter gather and map Renamed the values that the rename dist keyword argument can have from basic to b 190 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Python has a larger set of dependencies if you want all of its capabilities See the setup py script for details e The constructors for IPython kernel client MultiEngineClient and IPython kernel client TaskClient no longer take the ip port tuple Instead they take the filename of a file that contains the FURL for that client If the FURL file is in your IPYTHONDIR it will be found automatically and the constructor can be left empty The asynchronous clients in TPython kernel asyncclient are now created using the factory functions get_multiengine_client and get_task_client These return a Deferred to the actual client The command line options to ipcontroller and ipengine have changed to reflect the new Foolscap network protocol and the FURL files Please see the help for these scripts for details The configuration files for the kernel have changed because of the Foolscap stuff If you were using custom config files before you should delete them and regenerate new ones Changes merged in from IPython1 New features Much improved setup py and setupegg py sc
364. l Test failure in Python parallel IPEP1 Cell magics and general cleanup of the Magic system Refactoring of the magics system and implementation of cell magics test_pr should clearn PY THONPATH for the subprocesses Merge divergent Kernel implementations notebook Make pager resizable and remember size Share code for pycat and loadpy make pycat aware of URLs Adding interactive inline plotting to notebooks with flot notebook Open HTML links in a new window by default Open Python notebook hyperlinks in a new window using target _blank Open Python notebook hyperlinks in a new window using target _blank Fix typo enconters gt encounters Clear window title when kernel is restarted 735 Images missing from XML SVG export for me 1449 1752 Fix for bug 735 Images missing from XML SVG export Reconnect Websocket when it closes unexpectedly Reconnect Websocket when it closes unexpectedly Load MathJax js using HTTPS when Python notebook server is HTTPS Tooltip completer js refactor 112 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 1700 A module for sending custom user messages from the kernel 1745 htmlnotebook Cursor is off 1728 ipython crash with matplotlib during picking 1681 add qt config option to clear_on_kernel_restart 1733 Tooltip completer js refactor 1676 Kernel status shutdown from dashboard 1658 Alternate notebook upload
365. l exporters open markdown links in new tabs remove mention of nonexistent _margv in macro set overflow x hidden on Firefox only Fix missing import os in latex exporter Don t hardcode latex posix paths in nbconvert fix path prefix in nbconvert Use a CDN to get reveal js library When a notebook is written to file name the metadata name u Change to standard save icon in Notebook toolbar 30 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 3539 PR 3508 PR 3526 PR 3525 PR 3532 PR 3515 PR 3513 PR 3536 PR 3521 PR 3530 PR 3528 PR 3518 PR 3506 PR 3516 PR 3523 PR 3514 PR 3520 PR 3507 PR 3512 PR 3511 PR 3510 PR 3504 PR 3500 PR 3478 PR 3493 PR 3488 PR 3484 PR 3481 PR 3468 PR 3430 PR 3491 PR 3485 Don t hardcode posix paths in nbconvert notebook supports raw_input and debug now ensure default is first in cluster profile list basic timezone info include nbconvert templates in installation update CodeMirror component to 3 14 add No Checkpoints to Revert menu format positions are required in Python 2 6 x Nbconvert fix silent fail if template doesn t exist update store magic docstring fix local mathjax with custom base_project_url Clear up unused imports store r restores saved aliases and directory history as well as variables make css highlight style configu
366. l reconnect the web application to the same kernel In the dashboard notebooks with an active kernel have a Shutdown button next to them whereas notebooks without an active kernel have a Delete button in its place Other clients may connect to the same underlying Python kernel The notebook server always prints to the terminal the full details of how to connect to each kernel with messages such as the following NotebookApp Kernel started 87 7d2c0 13e3 43df 8bb8 1bd37aaf3373 This long string is the kernel s ID which is sufficient for getting the information necessary to connect to the kernel You can also request this connection data by running the connect_info magic This will print the same ID information as well as the content of the JSON data structure it contains You can then for example manually start a Qt console connected to the same kernel from the command line by passing a portion of the ID ipython qtconsole existing 87f 7d2c0 Without an ID existing will connect to the most recently started kernel This can also be done by running the qt console magic in the notebook See also 246 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack Decoupled two process model 4 6 3 Notebook user interface When you create a new notebook document you will be presented with the notebook name a menu bar a toolbar and an empty
367. lator program Emacs users see below To permanently set your color preferences edit the file IPYTHONDIR ipythonrc and set the colors option to the desired value 6 7 3 Object details types docstrings source code etc Python has a set of special functions for studying the objects you are working with discussed in detail here But this system relies on passing information which is longer than your screen through a data pager such as the common Unix less and more programs In order to be able to see this information in color your pager needs to be properly configured I strongly recommend using less instead of more as it seems that more simply can not understand colored text correctly In order to configure less as your default pager do the following 1 Set the environment PAGER variable to less 2 Set the environment LESS variable to r plus any other options you always want to pass to less by default This tells less to properly interpret control sequences which is how color information is given to your terminal For the bash shell add to your bashrc file the lines export PAGER less export LESS r For the csh or tcsh shells add to your cshre file the lines setenv PAGER less setenv LESS r There is similar syntax for other Unix shells look at your system documentation for details If you are on a system which lacks proper data pagers such as Windows Python will use a very l
368. ler and engines on a single host using the ipcluster command To start a controller and 4 engines on your localhost just do 5 1 Overview and getting started 263 IPython Documentation Release 1 2 1 An Afternoon Hack ipcluster start n 4 More details about starting the Python controller and engines can be found here Once you have started the Python controller and one or more engines you are ready to use the engines to do something useful To make sure everything is working correctly try the following commands In 1 from IPython parallel import Client In 2 Client In 4 c ids Out 4 set 0 1 2 3 In 5 c apply_sync lambda Hello World Out 5 Hello World Hello World Hello World Hello World When a client is created with no arguments the client tries to find the corresponding JSON file in the local ipython profile_default security directory Or if you specified a profile you can use that with the Client This should cover most cases In 2 c Client profile myprofile If you have put the JSON file in a different location or it has a different name create the client like this In 2 c Client path to my ipcontroller client json Remember a client needs to be able to see the Hub s ports to connect So if they are on a different machine you may need to use an ssh server to tunnel a
369. ler solves this problem to expose a clean asynchronous API to the user IPython controller The Python controller processes provide an interface for working with a set of engines At a general level the controller is a collection of processes to which Python engines and clients can connect The controller is composed of a Hub and a collection of Schedulers These Schedulers are typically run in separate processes but on the same machine as the Hub but can be run anywhere from local threads or on remote machines The controller also provides a single point of contact for users who wish to utilize the engines connected to the controller There are different ways of working with a controller In IPython all of these models 260 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack 5 1 Overview and getting started 261 IPython Documentation Release 1 2 1 An Afternoon Hack are implemented via the View apply method after constructing View objects to represent subsets of engines The two primary models for interacting with engines are e A Direct interface where engines are addressed explicitly e A LoadBalanced interface where the Scheduler is trusted with assigning work to appropriate en gines Advanced users can readily extend the View models to enable other styles of parallelism Note A single controller and set of engines can be used with multiple models
370. les ftp pi super computing org 2 pi200m that each have 10 million digits of pi For the parallel calculation we have copied these files to the local hard drives of the compute nodes A total of 15 of these files will be used for a total of 150 million digits of pi To make things a little more interesting we will calculate the frequencies of all 2 digits sequences 00 99 and then plot the result using a 2D matrix in Matplotlib The overall idea of the calculation is simple each Python engine will compute the two digit counts for the digits in a single file Then in a final step the counts from each engine will be added up To perform this calculation we will need two top level functions from pidigits py def compute_two_digit_freqs filename mon Read digits of pi from a file and compute the 2 digit frequencies 5 11 Parallel examples 323 IPython Documentation Release 1 2 1 An Afternoon Hack mN d txt_file_to_digits filename freqs two_digit_freqs d return fregs def reduce_freqs freqlist mon Add up a list of freq counts to get the total counts nN allfreqs np zeros_like freqlist 0 for f in freqlist allfreqs f return allfreqs We will also use the plot_two_digit_freqs function to plot the results The code to run this calculation in parallel is contained in examples parallel parallelpi py This code can be run in parallel using Python by following these steps
371. les test_mono wav Move Up Down And we can easily view its spectral structure using matplotlib s builtin specgram routine Run Selected All Autoindent i In 3 fig ax1 ax2 plt subplots 1 2 figsize 12 4 ax1 plot x ax1 set_title Raw audio signal Kernel ax2 specgram x ax2 set_title Spectrogram Actions interrupt Restart Raw audio signal Spectrogram Kill kernel upon exit Help Links Python IPython NumPy SciPy MPL SymPy Shift Enter run selected cell Ctri Enter run in terminal mode Ctri m h show keyboard shortcuts 3 10000 20000 30000 40000 50000 5000 10000 15000 20000 25000 Fig 2 3 The new IPython notebook showing text mathematical expressions in LaTeX code results and embedded figures created with Matplotlib This Notebook is a major milestone for IPython as for years we have tried to build this kind of system We were inspired originally by the excellent implementation in Mathematica we made a number of attempts using older technologies in earlier Summer of Code projects in 2005 both students and Robert Kern devel oped early prototypes and in recent years we have seen the excellent implementation offered by the Sage lt http sagemath org gt system But we continued to work on something that would be consistent with the rest of IPython s design and it is clear now that the effort was worth it based on the ZeroMQ communica tions architecture introduc
372. lient over the shell channel to be made available to all clients in a uniform manner All messages are tagged with enough information details below for clients to know which messages come from their own interaction with the kernel and which ones are from other clients so they can display each type appropriately The actual format of the messages allowed on each of these channels is specified below Messages are dicts of dicts with string keys and values that are reasonably representable in JSON Our current implementation uses JSON explicitly as its message format but this shouldn t be considered a permanent feature As we ve discovered that JSON has non trivial performance issues due to excessive copying we may in the future move to a pure pickle based raw message format However it should be possible to easily convert from the raw objects to JSON since we may have non python clients e g a web frontend As long as it s easy to make a JSON version of the objects that is a faithful representation of all the data we can communicate with such clients Note Not all of these have yet been fully fleshed out but the key ones are see kernel and frontend files for actual implementation details 7 2 2 General Message Format A message is defined by the following four dictionary structure The message header contains a pair of unique identifiers for the originating session and the actual message id in addition
373. ling back some results This can all be done with a single function Let s say you want to compute the dot product of two matrices one of which resides on the engine and another resides on the client You might construct a task that looks like this In 10 st kclient StringTask import numpy C numpy dot A B Ww T push dict B B pull c In 11 tid tc run st In 12 tr tce get_task_result tid In 13 C tc C In the new code this is simpler 344 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack In 10 import numpy In 11 from IPython parallel import Reference In 12 ar lbview apply numpy dot Reference A B In 13 C ar get Note the use of Reference This is a convenient representation of an object that exists in the engine s namespace so you can pass remote objects as arguments to your task functions Also note that in the kernel model after the task is run A B and C are all defined on the engine In order to deal with this there is also a clear_after flag for Tasks to prevent pollution of the namespace and bloating of engine memory This is not necessary with the new code because only those objects explicitly pushed or set via globals will be resident on the engine beyond the duration of the task See also Dependencies also work very differ
374. ll Message type input_request 400 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack content prompt str Message type input_reply content value str Note We do not explicitly try to forward the raw sys stdin object because in practice the kernel should behave like an interactive program When a program is opened on the console the keyboard effectively takes over the st din file descriptor and it can t be used for raw reading anymore Since the Python kernel effectively behaves like a console program albeit one whose keyboard is actually living in a separate process and transported over the zmq connection raw stdin isn t expected to be available 7 2 8 Heartbeat for kernels Initially we had considered using messages like those above over ZMQ for a kernel heartbeat a way to detect quickly and reliably whether a kernel is alive at all even if it may be busy executing user code But this has the problem that if the kernel is locked inside extension code it wouldn t execute the python heartbeat code But it turns out that we can implement a basic heartbeat with pure ZMQ without using any Python messaging at all The monitor sends out a single zmq message right now it is a str of the monitor s lifetime in seconds and gets the same message right back prefixed with the zmq identity of the DEALER sock
375. ll stdout 0 stdin_port 60387 ripta L127 0 0u 1 hb_port 50835 key 2dd69 7dd3 4340 bf3e 7e2e22a62542 Shell port 55328 iopub_port 58264 Paste the above JSON into a file and connect with gt ipython lt app gt existing lt file gt or if you are local you can connect with just gt ipython lt app gt existing kernel 60125 json or even just gt ipython lt app gt existing if this is the most recent IPython session you have started stdout 1 stdin port 61869 Note qtconsole will call bind_kernel on an engine if it hasn t been done already so you can often skip that first step 5 5 The IPython task interface The task interface to the cluster presents the engines as a fault tolerant dynamic load balanced system of workers Unlike the multiengine interface in the task interface the user have no direct access to individual engines By allowing the IPython scheduler to assign work this interface is simultaneously simpler and more powerful Best of all the user can use both of these interfaces running at the same time to take advantage of their respective strengths When the user can break up the user s work into segments that do not depend on previous execution the task interface is ideal But it also has more power and flexibility allowing the user to guide the distribution of jobs without having to assign tasks to engines explicitly 5 5 1 Starting the
376. ll1 instance _ip and __IP have been removed from the user s namespace They are replaced by a new function called get_ipython that re turns the current InteractiveShe11 instance This function is injected into the user s namespace and is now the main way of accessing the running Python Old style configuration files ipythonrc and ipy_user_conf py are no longer supported Users should migrate there configuration files to the new format described here and here The old Python extension API that relied on ipapi has been completely removed The new extension API is described here Support for qt 3 has been dropped Users who need this should use previous versions of Python 2 8 0 11 Series 161 IPython Documentat ion Release 1 2 1 An Afternoon Hack e Removed shellglobals as it was obsolete e Removed all the threaded shells in TPython core shell These are no longer needed because of the new capabilities in TPython lib inputhook e New top level IPython util sub packages have been created s IPython deathrow IPython quarantine IPython core IPython lib All existing top level modules have been moved to appropriate sub packages All internal import statements have been updated and tests this section of the e IPython ipap IPython ipli have been added The build system setup py and friends have been updated See documentation for descriptions of these new sub packages i
377. luding HTML for example for blog posts reStructeredText LaTeX PDF and slide shows via the new nbconvert command Furthermore any ipynb notebook document available from a public URL can be shared via the Python Notebook Viewer nbviewer This service loads the notebook document from the URL and renders it as a static web page The results may thus be shared with a colleague or as a public blog post without other users needing to install Python themselves In effect nbviewer is simply nbconvert as a web service so you can do your own static conversions with nbconvert without relying on nbviewer See also Details on the notebook JSON file format 4 6 2 Starting the notebook server You can start running a notebook server from the command line using the following command ipython notebook This will print some information about the notebook server in your console and open a web browser to the URL of the web application by default http 127 0 0 1 8888 4 6 The IPython Notebook 245 IPython Documentation Release 1 2 1 An Afternoon Hack The landing page of the Python notebook web application the dashboard shows the notebooks currently available in the notebook directory by default the directory from which the notebook server was started You can create new notebooks from the dashboard with the New Notebook button or open existing ones by clicking on their name You can also drag and drop ipynb
378. lues PESE cmap gist_rainbow vy Ni wA GY ZA NA f Fig 5 1 Time started on x runtime on y and color coded by engine id in this case there were four engines Edges denote dependencies 5 13 Details of Parallel Computing with IPython Note There are still many sections to fill out in this doc 5 13 1 Caveats First some caveats about the detailed workings of parallel computing with OMQ and IPython 5 13 Details of Parallel Computing with IPython 331 IPython Documentation Release 1 2 1 An Afternoon Hack Non copying sends and numpy arrays When numpy arrays are passed as arguments to apply or via data movement methods they are not copied This means that you must be careful if you are sending an array that you intend to work on PyZMQ does allow you to track when a message has been sent so you can know when it is safe to edit the buffer but Python only allows for this It is also important to note that the non copying receive of a message is read only That means that if you intend to work in place on an array that you have sent or received you must copy it This is true for both numpy arrays sent to engines and numpy arrays retrieved as results The following will fail In 3 A numpy zeros 2 In 4 def setter a a 0 1 return a In 5 rc 0 apply_sync setter A RuntimeError Traceback most recent call last lt sty lt ipytho
379. ly more version agnostic Fix for issue 56 Adding m option to run similar to m for python interpreter promote aliases and flags to ensure they have priority over config files BUG Completion widget position and pager focus Allow connection to kernels by files Two process terminal frontend make sdist flags work again e g manifest only Add Tab key to list of keys that scroll down the paging widget Fix for issue 800 Python3 setup py install failiure 2 7 Issues closed in the 0 12 development cycle 141 IPython Documentation Release 1 2 1 An Afternoon Hack 845 Tests on Python 3 802 DOC extensions add documentation for the bundled extensions 830 contiguous stdout stderr in notebook 761 Windows test runner fails if repo path e g home dir contains spaces 801 Py3 notebook 809 use CFRunLoop directly in ipython kernel pylab osx 841 updated old scipy org links other minor doc fixes 837 remove all trailling spaces 834 Issue https github com ipython ipython issues 832 resolution 746 ENH extensions port autoreload to current API 828 fixed permissions sub modules should not be executable added shebang for run_ipy_in_profiler py 798 pexpect amp Python 3 804 Magic range crash if greater than len input_hist 821 update tornado dependency to 2 1 807 Faciliate ssh tunnel sharing by announcing ports 795 Add cluster id for multiple cluster instances per profile 742 Glut 6
380. m and use the LRU of the two This is known to be better than plain random in many cases but requires a small amount of computation leastload Least Load This is the default scheme Always assign tasks to the engine with the fewest outstanding tasks LRU breaks tie weighted Weighted Two Bin Random Requires numpy Pick two engines at random using the number of outstanding tasks as inverse weights and use the one with the lower load Greedy Assignment Tasks can be assigned greedily as they are submitted If their dependencies are met they will be assigned to an engine right away and multiple tasks can be assigned to an engine at a given time This limit is set with the TaskScheduler hwm high water mark configurable in your ipcontroller_config py config file with the most common choices are c TaskSheduler hwm 0 minimal latency default in IPython lt 0 13 or c TaskScheduler hwm 1 most informed balancing default in 0 13 In IPython lt 0 13 the default is 0 or no limit That is there is no limit to the number of tasks that can be outstanding on a given engine This greatly benefits the latency of execution because network traffic can be hidden behind computation However this means that workload is assigned without knowledge of how long each task might take and can result in poor load balancing particularly for submitting a collection of heterogeneous tasks all at once You can limit this eff
381. magic cd command along with a persistent bookmark system using bookmark for fast access to frequently visited directories A lightweight persistence framework via the store command which allows you to save arbitrary Python variables These get restored when you run the store r command Automatic indentation optional of code as you type through the readline library Macro system for quickly re executing multiple lines of previous input with a single name via the macro command Macros can be stored persistently via store and edited via edit Session logging you can then later use these logs as code in your programs Logs can optionally timestamp all input and also store session output marked as comments so the log remains valid Python source code Session restoring logs can be replayed to restore a previous session to the state where you left it Verbose and colored exception traceback printouts Easier to parse visually and in verbose mode they produce a lot of useful debugging information basically a terminal version of the cgitb module Chapter 1 Introduction IPython Documentation Release 1 2 1 An Afternoon Hack e Auto parentheses via the Sautocall command callable objects can be executed without parenthe ses Sin 3 is automatically converted to sin 3 e Auto quoting using or as the first character forces auto quoting of the rest of the line my_function a b becomes automatically my_function a
382. magic does not work in Python 3 Issues with pyzmq and ipython on EPD update 54 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 2434 2720 2234 2842 3244 Ystore magic not auto restoring base_url and static path Update various low resolution graphics for retina displays Remember passwords for pw protected notebooks qtconsole ValueError close_fds is not supported on Windows platforms if you redirect stdin stdout stderr 2215 2268 1905 2380 3185 2821 2475 2470 860 2652 1587 folder 2710 2596 2671 2703 2803 2693 2531 2520 2279 2161 1292 1731 3227 2896 3517 AsyncResult wait 0 can hang waiting for the client to get results provide mean to retrieve static data path Expose UI for worksheets within each notebook Qt inputhook prevents modal dialog boxes from displaying prettify on double Test failure Python parallel tests test_client test_resubmit_header Notebook Line is deindented when typing eg a colon in markdown mode Do not destroy valid notebooks Allow the standalone export of a notebook to HTML notebook with qt backend crashes at save image location popup Improve kernel restarting in the notebook Saving a plot in Mac OS X backend crashes Python notebook Last saved is misleading on file opening TypeError NoneType when executed ipytho
383. magic needs to use self shell DirectView map_sync with Lambdas Using Generators 2 3 Issues closed in the 1 0 development cycle 69 IPython Documentation Release 1 2 1 An Afternoon Hack 2091 2132 2156 2154 2162 2157 2151 2139 2143 2135 2093 2122 2120 1997 2129 2121 2114 2085 2119 2052 2111 2112 2108 1425 1513 2087 2103 2082 2083 2094 2077 1760 Unable to use print_stats after prun r in notebook Add fail over for pastebin Make it possible to install ipython without nasty gui dependencies Scrolled long output should be off in print view by default Tab completion does not work with Python embed_kernel iPython 0 13 github master cannot create logfile from scratch missing newline when a magic is called from the qtconsole menu 00_notebook_tour Image example broken on master Add a cython_annotate magic Running Python from terminal Makefile for building Sphinx documentation on Windows Bug in pretty printing Notebook Make a Copy keeps opening duplicates in the same tab password cannot be used with url prefix help doc displayed multiple times if requested in loop ipdb does not support input history in qtconsole logstart doesn t log ed magic fails in qtconsole iPython fails to run on MacOS Lion pylab inline magic does not work on windows Ipython won t start on W7 Strange internal traceback Backslash at t
384. mbedding IPython You can start a regular Python session with import IPython IPython start_ipython at any point in your program This will load Python configuration startup files and everything just as if it were a normal Python session In addition to this it is possible to embed an Python instance inside your own Python programs This allows you to evaluate dynamically the state of your code operate with your variables analyze them etc Note however that any changes you make to values while in the shell do not propagate back to the running code so it is safe to modify your values because you won t break your code in bizarre ways by doing so Note At present embedding Python cannot be done from inside Python Run the code samples below outside Python This feature allows you to easily have a fully functional python environment for doing object introspection anywhere in your code with a simple function call In some cases a simple print statement is enough but if you need to do more detailed analysis of a code fragment this feature can be very valuable 220 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack It can also be useful in scientific computing situations where it is common to need to do some automatic computationally intensive part and then stop to look at data plots etc Opening an Python instance will give you full acce
385. mes later 6 6 Custom input transformation Python extends Python syntax to allow things like magic commands and help with the syntax There are several ways to customise how the user s input is processed into Python code to be executed These hooks are mainly for other projects using Python as the core of their interactive interface Using them carelessly can easily break Python 6 6 1 String based transformations When the user enters a line of code it is first processed as a string By the end of this stage it must be valid Python syntax These transformers all subclass TPython core inputtransformer InputTransformer and are used by IPython core inputsplitter IPythonInputSplitter These transformers act in three groups stored separately as lists of instances in attributes of IPythonInputSplitter e physical_line_transforms act on the lines as the user enters them For example these strip Python prompts from examples pasted in logical_line_transforms act on lines as connected by explicit line continuations i e at the end of physical lines They are skipped inside multiline Python statements This is the point where Python recognises smagic commands for instance python_line_transforms act on blocks containing complete Python statements Multi line strings lists and function calls are reassembled before being passed to these but note that function and class definitions are still a series of separate s
386. middle click paste broken in notebook 1670 Point pastebin to gist 1667 Test failure in test_message_spec 114 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 213 1668 Test failure in Python zmq tests test_message_spec test_complete pyout status 1669 handle pyout messages in test_message_spec 1295 add binary tree engine interconnect example 1642 Cherry picked commits from 0 12 1 release 1659 Handle carriage return characters r in HTML notebook output 1313 Figure out MathJax 2 support 1653 Test failure in Python zmq 1656 ensure kernels are cleaned up in embed_kernel tests 1666 pip install ipython dev installs version 0 8 from an old svn repo 1664 InteractiveShell run_code Update docstring 1512 print stuff should avoid newline 1662 Delay flushing softspace until after cell finishes 1643 handle jpg jpeg in the qtconsole 966 dreload fails on Windows XP with iPython 0 11 Unexpected Error 1500 dreload doesn t seem to exclude numpy 1520 kernel crash when showing tooltip 1652 add patch_pyzmq for backporting a few changes from newer pyzmq 1650 DOC moving files with SSH launchers 1357 add IPython embed_kernel 1640 Finish up embed_kernel 1651 Remove bundled Itpl module 1634 incremental improvements to SSH launchers 1649 move examples test_embed into ex
387. mmands The four most helpful commands as well as their brief description is shown to you in a banner every time you start Python command description Introduction and overview of Python s features Yoquickref Quick reference help Python s own help system object Details about object use object for extra details 4 1 2 Tab completion Tab completion especially for attributes is a convenient way to explore the structure of any object you re dealing with Simply type ob jJect_name lt TAB gt to view the object s attributes see the readline section for more Besides Python objects and keywords tab completion also works on file and directory names 4 1 3 Exploring your objects Typing object_name will print all sorts of details about any object including docstrings function definition lines for call arguments and constructor details for classes To get specific information on an object you can use the magic commands pdoc pdef psource and pfile 203 IPython Documentation Release 1 2 1 An Afternoon Hack 4 1 4 Magic functions Python has a set of predefined magic functions that you can call with a command line style syntax There are two kinds of magics line oriented and cell oriented Line magics are prefixed with the character and work much like OS command line calls they get as an argument the rest of the line where arguments are passed witho
388. mo The following listing shows the markup necessary for dividing a script into sections for execution as a demo coding utf 8 4 A simple interactive demo to illustrate the use of IPython s Demo class Any python script can be run as a demo but that does little more than showing it on screen syntax highlighted in one shot If you add a little simple markup you can stop at specified intervals and return to the ipython prompt resuming execution later This is a unicode test moe from future import print_function print Hello welcome to an interactive IPython demo print Executing this block should require confirmation before proceeding print unless auto_all has been set to true in the demo object The mark below defines a block boundary which is a point where IPython will stop execution and return to the interactive prompt Note that in actual interactive execution lt demo gt stop x 1 y 2 lt demo gt stop the mark below makes this block as silent lt demo gt silent print This is a silent block which gets executed but not printed lt demo gt stop lt demo gt auto print This is an automatic block print It is executed without asking for confirmation but printed Z xty print z x lt demo gt stop This is just another normal block print z is now Z print bye In order t
389. mory at the obvious expense of being able to access the information that would have been stored in the database used for task resubmission requesting results of tasks you didn t submit etc To use this backend simply pass nodb to ipcont roller on the command line or specify the NoDB class in your ipcontroller_config py as described above See also For more information on the database backends see the db backend reference Configuring ijpengine The Python Engine takes its configuration from the file ipengine_config py The Engine itself also has some amount of configuration Most of this has to do with initializing MPI or connecting to the controller To instruct the Engine to initialize with an MPI environment set up by mpi4py add 5 2 Starting the IPython controller and engines 277 IPython Documentation Release 1 2 1 An Afternoon Hack c MPI use mpidpy In this case the Engine will use our default mpi4py init script to set up the MPI environment prior to exection We have default init scripts for mpi4py and pytrilinos If you want to specify your own code to be run at the beginning specify c MPI init_script You can also specify a file or python command to be run at startup of the Engine c IPEngineApp startup_script u path to my startup py c IPEngineApp startup_command import numpy scipy mpi4py These commands files will be run again after each It
390. mples supplied which we will briefly describe now These can be used as is and both provide very useful functionality or you can use them as a starting point for writing your own extensions Pasting of code starting with Python or IPython prompts Python is smart enough to filter out input prompts be they plain Python ones gt gt gt and or Python ones In N and You can therefore copy and paste from existing interactive sessions without worry The following is a screenshot of how things work copying an example from the standard Python tutorial In 1 gt gt gt Fibonacci series In 2 the sum of two elements defines the next In S ess ay b 0 1 In 4 gt gt gt while b lt 10 print b a b b atb Do 0 w e he jH And pasting from IPython sessions works equally well In 1 In 5 def f 8 A simple function return xX 2 Out 2 9 226 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack 4 3 7 GUI event loop support New in version 0 11 The gui magic and I Python 1lib inputhook Python has excellent support for working interactively with Graphical User Interface GUD toolkits such as wxPython PyQt4 PySide PyGTK and Tk This is implemented using Python s builtin PyOS Input Hook hook This implementation is extremely robust compared to our previous thread bas
391. n Note This section is not thorough and Python kernel zmq needs a thorough security audit 310 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack IPython s IPython kernel zmq package exposes the full power of the Python interpreter over a TCP IP network for the purposes of parallel computing This feature brings up the important question of Python s security model This document gives details about this model and how it is implemented in Python s architecture 5 9 1 Process and network topology To enable parallel computing Python has a number of different processes that run These processes are discussed at length in the Python documentation and are summarized here e The Python engine This process is a full blown Python interpreter in which user code is executed Multiple engines are started to make parallel computing possible The Python hub This process monitors a set of engines and schedulers and keeps track of the state of the processes It listens for registration connections from engines and clients and monitor connections from schedulers The Python schedulers This is a set of processes that relay commands and results between clients and engines They are typically on the same machine as the controller and listen for connections from engines and clients but connect to the Hub The Python client This process is typically an interactive Pyth
392. n 2 kernels nbconvert Fix coalescestreams line with incorrect nesting causing strange behavior only strip continuation prompts if regular prompts seen first Fixed name error bug in function safe_unicode in module py3compat move test_kernel from Python zmq to Python kernel ZMQ heartbeat channel catch EINTR exceptions and continue use unicode for HTML export fix a couple of default block values Update docs on declaring a magic function restore accidentally removed EngineError minor docs changes respect pylab_import_all when pylab specified at the command line Make Qt console banner configurable 2 3 Issues closed in the 1 0 development cycle 17 IPython Documentation Release 1 2 1 An Afternoon Hack PR 4086 fix missing errno import PR 4030 exclude git in MANIFEST in PR 4047 Use istypeQ when checking if canned object is a dict PR 4031 don t close_fds on Windows PR 4029 bson Binary moved PR 4035 Fixed custom jinja2 templates being ignored when setting template_path PR 4026 small doc fix in nbconvert PR 4016 Fix Python start_ functions PR 4021 Fix parallel client View map on numpy arrays PR 4022 DOC fix links to matplotlib notebook docs PR 4018 Fix warning when running IPython kernel tests PR 4019 Test skipping without unicode paths PR 4008 Transform code before prun prun runs PR 4014 Fix typo in ipapp PR 3987 get files list in backport_pr PR 3974 n
393. n Tips amp Tricks 207 ol at work IPython Documentation Release 1 2 1 An Afternoon Hack This section was written after a contribution by Alexander Belchenko on the Python user list 4 3 IPython reference 4 3 1 Command line usage You start Python with the command ipython options files Note For Python on Python 3 use ipython3 in place of ipython If invoked with no options it executes all the files listed in sequence and drops you into the interpreter while still acknowledging any options you may have set in your ipython_config py This behavior is different from standard Python which when called as python i will only execute one file and ignore your configuration setup Please note that some of the configuration options are not available at the command line sim ply because they are not practical here Look into your configuration files for details on those There are separate configuration files for each profile and the files look like ipython_config py or ipython_config_ lt frontendname gt py Profile directories look like profile_profilename and are typically installed in the IPYTHONDIR directory For Linux users this will be HOME config ipython and for other users it will be HOME ipython For Windows users HOME resolves to C Documents and Set tings YourUserName in most instances Eventloop integration Previously Python had command line options for controllin
394. n calls edited dochelp Don t ask for confirmation when stdin isn t available Cell level cut amp paste overwrites multiple cells Minor but important fixes to cut copy paste TaskScheduler hwm default value TaskScheduler hwm default to 1 instead of 0 in Hub registration_timeout must be an integer but heartmonitor period is CFloat HeartMonitor period should be an Integer Allow merge split adjacent cells in notebook Aceify Mergesplit Another strange input handling error a fix for GH 1269 Dead kernel loop ImportError cannot import name S1 from logging notebook menu item to send a KeyboardInterrupt to the kernel BUG Minor typo in history_console_widget py Python notebook doesn t work with lastest version of tornado add NoDB for non recording Hub allow Reference as callable in map apply 124 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 1257 1220 use self kernel_manager_class in qtconsoleapp Open a new notebook while connecting to an existing kernel opened by qtconsole or terminal or standalone 1253 1260 1262 1225 1229 1256 1246 1250 1244 1194 1198 1211 1224 1187 1207 1060 1066 1141 1231 1129 1199 1219 1218 1217 1216 1197 1175 1208 1186 set auto_create flag for notebook apps heartbeat failure on long gil holding operation Heartbeat no longer shares the app s Co
395. n input 12 c3e7afeb3075 gt in setter a RuntimeError array is not writeable If you do need to edit the array in place just remember to copy the array if it s read only The ndarray flags writeable flag will tell you if you can write to an array In 3 A numpy zeros 2 In 4 def setter a only copy read only arrays if not a flags writeable a a copy a 0 1 return a www In 5 rc 0 apply_sync setter A Out 5 array 1 0 1 note that results will also be read only In 6 _ flags writeable Out 6 False If you want to safely edit an array in place after sending it you must use the track True flag IPython always performs non copying sends of arrays which return immediately You must instruct IPython track those messages at send time in order to know for sure that the send has completed AsyncResults have a sent property and wait_on_send method for checking and waiting for OMQ to finish with a buffer In 5 A numpy random random 1024 1024 In 6 view track True 332 Chapter 5 Using IPython for parallel computing ing gt in lt mo IPython Documentation Release 1 2 1 An Afternoon Hack In 7 ar view apply_async lambda x 2x x A In 8 ar sent Out 8 False In 9 ar wait_on_send blocks until sent is True What is sendable If Python doesn t know what to do with an object
396. n ipython 0 13 x python 3 3 no warning when leaving closing notebook on master without saved changes No obvious feedback when kernel crashes Move all autoreload tests to their own group sh py and ipython for python 3 3 Yomagic doesn t work Editing markdown cells that wrap has off by one errors in cursor positioning Python for Python3 unexpected keyword argument to remove_kernel TypeError when using after a string in a logstart session rmagic and pandas DataFrame Ellipsis literal triggers AttributeError Test unicode source in pinfo drop Python 3 1 support IPEP 2 Input transformations Yopaste and cpaste not removing lines Testing fails because iptest py cannot be found will not run within PIL build directory RMagic Dataframe Conversion Problem Empty lines disappear from triple quoted literals Broken link on Python core display page Config not passed down to subcommands Setting log_format in config file results in error and no format changes Notebook occasionally not working on windows Sophos AV Cursor positioning off by a character in auto wrapped lines 56 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 3454 3457 3447 1373 1504 1459 3001 3077 3056 3120 3156 3197 3309 3415 3446 3295 3442 3438 1378 3437 3435 3424 3434 3426 3412 3408 3246 3404 3405 3259
397. n on this approach see our documentation on ipcluster 5 7 3 Actually using MPI Once the engines are running with MPI enabled you are ready to go You can now call any code that uses MPI in the Python engines And all of this can be done interactively Here we show a simple example that uses mpi4py mpi4py version 1 1 0 or later First lets define a simply function that uses MPI to calculate the sum of a distributed array Save the following text in a file called psum py from mpi4py import MPI import numpy as np def psum a locsum np sum a rcevBuf np array 0 0 d MPI COMM_WORLD Allreduce locsum MPI DOUBLE rcvBuf MPI DOUBLE op MPI SUM return rcvBuf Now start an Python cluster ipcluster start profile mpi n 4 Note It is assumed here that the mpi profile has been set up as described here Finally connect to the cluster and use this function interactively In this case we create a distributed array and sum up all its elements in a distributed manner using our psum function In 1 from IPython parallel import Client In 2 c Client profile mpi 5 7 Using MPI with IPython 307 IPython Documentation Release 1 2 1 An Afternoon Hack In 3 view c In 4 view activate enable magics run the contents of the file on each engin In 5 view run psum py In 6 view scatter a np arange 16 dtype
398. n qtconsole in windows console Notebook scrolling breaks after pager is shown KernelManager and KernelClient should be two separate objects TerminallPythonApp configuration fails without ipython_config py Python 0 13 1 python 2 32 bit installer includes 64 bit ipython exe launchers in the scripts Control C kills port forwarding Setting __file__ to None breaks Mayavi import When logged into notebook long titles are incorrectly positioned Notebook Print view should not be editable test parallel launchers Improve documentation of ipcontroller and possible BUG IPController very unstable documentation build broken in head 2 3 Issues closed in the 1 0 development cycle 55 IPython Documentation Release 1 2 1 An Afternoon Hack 3522 3448 3519 3345 3509 3471 3502 3302 3475 3490 1912 2577 3467 3501 3492 3474 2283 2787 2605 1179 2055 2293 2790 3480 2580 2797 2838 3050 3473 3462 3311 3461 UnicodeDecodeError ascii codec can t decode byte on Pycharm on Windows Please include MathJax fonts with Python Notebook Python Parallel map mysteriously turns pandas Series into numpy ndarray IPython embedded shells ask if I want to exit but I set confirm_exit False Python won t close without asking Are you sure in Firefox Notebook jinja2 markupsafe depedencies in manual Notebook broken in master autoreload does not work i
399. n the cluster with ipcluster start profile pbs n 128 Additional configuration options can be found in the PBS section of ipcluster_config Note Due to the flexibility of configuration the PBS launchers work with simple changes to the template for other qsub using systems such as Sun Grid Engine and with further configuration in similar batch systems like Condor Using ipcluster in SSH mode The SSH mode uses ssh to execute ipengine on remote nodes and ipcontroller can be run remotely as well or on localhost Note When using this mode it highly recommended that you have set up SSH keys and are using ssh agent SSH for password less logins As usual we start by creating a clean profile ipython profile create parallel profile ssh To use this mode select the SSH launchers in ipcluster_config py c IPClusterEngines engine_launcher_class SSHEngineSetLauncher and if the Controller is also to be remote c IPClusterStart controller_launcher_class SSHControllerLauncher The controller s remote location and configuration can be specified 270 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack H Set the user and hostname for the controller c SSHControllerLauncher hostname controller example com c SSHControllerLauncher user os environ get USER username
400. n this manner we ensure zero conflicts with any system wide installation you may already have while minimizing external dependencies for new users In Python 0 10 We ship argparse version 1 0 An improved and much more robust test suite that runs groups of tests in separate subprocesses using either Nose or Twisted s trial runner to ensure proper management of Twisted using code The test suite degrades gracefully if optional dependencies are not available so that the iptest command can be run with only Nose installed and nothing else We also have more and cleaner test decorators to better select tests depending on runtime conditions do setup teardown etc The new ipcluster now has a fully working ssh mode that should work on Linux Unix and OS X Thanks to Vishal Vatsa for implementing this The wonderful TextMate editor can now be used with edit on OS X Thanks to Matt Foster for this patch The documentation regarding parallel uses of IPython including MPI and PBS has been significantly updated and improved 186 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack The developer guidelines in the documentation have been updated to explain our workflow using bzr and Launchpad Fully refactored ipcluster command line program for starting Python clusters This new version is acomplete rewrite and 1 is fully cross platform we now use Twisted s process management 2 has much im
401. n version number optional Non python kernel backend may not have this version number The last component is an extra field which may be dev or rcl in development version It is an empty string for released version ipython_version int int int str Language version number mandatory It is Python version number e g 2 7 3 for the kernel included in IPython language_version int Programming language in which kernel is implemented mandatory Kernel included in IPython returns python language str 396 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack Kernel shutdown The clients can request the kernel to shut itself down this is used in multiple cases e when the user chooses to close the client application via a menu or window control e when the user types exit or quit or their uppercase magic equivalents e when the user chooses a GUI method like the Ctrl C shortcut in the IPythonQt client to force a kernel restart to get a clean kernel without losing client side state like history or inlined figures The client sends a shutdown request to the kernel and once it receives the reply message which is otherwise empty it can assume that the kernel has completed shutdown safely Upon their own shutdown client applications will typically execute a last minute sanity check and for
402. nagement features that the default windows console doesn t have So on Windows the installation procedure is 1 Install setuptools 2 Install pyreadline You can use the command easy_install pyreadline from a terminal or the binary installer appropriate for your platform from the PyPI page 3 Install Python itself which you can download from PyPI or from our site Note that on Windows 7 you must right click and Run as administrator for the Start menu shortcuts to be created Python by default runs in a terminal window but the normal terminal application supplied by Microsoft Windows is very primitive You may want to download the excellent and free Console application instead which is a far superior tool You can even configure Console to give you by default an Python tab which is very convenient to create new Python sessions directly from the working terminal 3 3 4 Installing the development version It is also possible to install the development version of Python from our Git source code repository To do this you will need to have Git installed on your system Then just do git clone recursive https github com ipython ipython git cd ipython S python setup py install Some users want to be able to follow the development branch as it changes If you have setuptools installed this is easy Simply replace the last step by S python setupegg py develop This creates links in the right
403. name of the namespace where the object was found builtin magics alias interactive etc namespace str The type name will be type __name__ for normal Python objects but it can also be a string like Magic function or System alias type_name str The string form of the object possibly truncated for length if detail_level is 0 String form str For objects with a __class__ attribute this will be set base_class str For objects with a __len__ attribute this will be set Length int 392 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack If the object is a function class or method whose file we can find we give its full path file str For pure Python callable objects we can reconstruct the object definition line which provides its call signature For convenience this is returned as a single definition field but below the raw parts that compose it are also returned as the argspec field vdefipition stir The individual parts that together form the definition string Clients with rich display capabilities may use this to provide a richer and more precise representation of the definition line e g by highlighting arguments based on the user s cursor position For non callable objects this field is empty argspec The names of all the arguments args list The name of the varargs ar
404. nc wait 3 for are done ist their results ar pr_list 0 get 2 998257160186 e ready using get or the 7676 2 9982588291168213 i in range 10 u r attribute 2 9987530708312988 2 99909990161895752 The block and targets keyword arguments and attributes Most DirectView methods excluding apply accept block and targets as keyword arguments As we have seen above these keyword arguments control the blocking mode and which engines the command is applied to The View class also has block and targets attributes that control the default behavior when the keyword arguments are not provided Thus the following logic is used for block and targets e If no keyword argument is provided the instance attributes are used e The Keyword arguments if provided overrides the instance attributes for the duration of a single call The following examples demonstrate how to use the instance attributes In 16 dview targets 0 2 5 3 IPython s Direct interface 283 IPython Documentation Release 1 2 1 An Afternoon Hack In 17 dview block False In 18 ar dview apply lambda 10 In 19 ar get Out 19 10 10 In 20 dview targets v client ids all engines 4 In 21 dview block True In 22 dview apply lambda 42 Out 22 42 42 42 42 The block and targets instance attributes of the DirectView also determine the be
405. nd In 14 This allows you to easily cut and paste multi line interactive prompts by printing them out they print like a clean string without prompt characters You can also manipulate them like regular variables they are strings modify or exec them typing exec _i9 will re execute the contents of input prompt 9 You can also re execute multiple lines of input easily by using the magic rerun or macro functions The macro system also allows you to re execute previous lines which include magic function calls which require special processing Type macro for more details on the macro system A history function hist allows you to see any part of your input history by printing a range of the _i variables You can also search grep through your history by typing hist g somestring This is handy for searching for URLs IP addresses etc You can bring history entries listed by hist g up for editing with the recall command or run them immediately with rerun Output caching system For output that is returned from actions a system similar to the input cache exists but using _ instead of _i Only actions that produce a result NOT assignments for example are cached If you are familiar with Mathematica Python s _ variables behave exactly like Mathematica s variables The following GLOBAL variables always exist so don t overwrite them e _ a single underscore stores previous output like Python s default
406. nd Thomas Kluyver started with our Python 3 port and by now has made major contributions to just about every area of Python We are also grateful for the support we have received during this development cycle from several institutions e Enthought Inc funded the development of our new Qt console an effort that required developing major pieces of underlying infrastructure which now power not only the Qt console but also our new parallel machinery We d like to thank Eric Jones and Travis Oliphant for their support as well as Ilan Schnell for his tireless work integrating and testing Python in the Enthought Python Distribution Nipy NIH funding via the NiPy project NIH grant 5RO1MH081909 02 helped us jumpstart the development of this series by restructuring the entire codebase two years ago in a way that would make modular development and testing more approachable Without this initial groundwork all the new features we have added would have been impossible to develop Sage NSF funding via the grant Sage Unifying Mathematical Software for Scientists Engineers and Mathematicians NSF grant DMS 1015114 supported a meeting in spring 2011 of several of the core Python developers where major progress was made integrating the last key pieces leading to this release 154 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e Microsoft s team working on Python Tools for Visual Studio develo
407. nderneath these methods call apply 336 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack In 51 dview a foo bar In 52 dview a Out 52 L foo bar foo bar foo bar foo bar Scatter and gather Sometimes it is useful to partition a sequence and push the partitions to different engines In MPI lan guage this is know as scatter gather and we follow that terminology However it is important to remember that in Python s Client class scatter is from the interactive Python session to the engines and gather is from the engines back to the interactive Python session For scatter gather operations be tween engines MPI should be used In 58 dview scatter a range 16 Out 58 None None None None In 59 dview a Out 59 O L 2 aly lar Be br Ty ye Se 20 Tl 12 13 14 5 J In 60 dview gather a Out 60 QO Ly 2 3 4r Sp 6 7 By Sp 10 Lil 127 13 7 14 15 Push and pull push pull Note TODO write this section LoadBalancedView The LoadBalancedView is the class for load balanced execution via the task scheduler These views always run tasks on exactly one engine but let the scheduler determine where that should be allowing load balancing of tasks The LoadBalanced View does allow you to specify
408. ndling take 2 cython_magic uses importlib which doesn t ship with py2 6 Remove importlib dependency which not available in Python 2 6 shell access using will not fill class or function scope vars Correct stack depth for variable expansion in system commands New JS completer should merge completions before display notebook deduplicate completion results no good error message on missing tkinter and paste Display message from TryNext error in magic_paste Remove args kwargs handling in TryNext fix paste error messages Keep line endings in ipynb Matplotlib window freezes using intreractive plot in qtconsole Improve CodeMagics _find_edit_target Colons in notebook name causes notebook deletion without warning Make invalid in filenames in the Notebook JS code doc cleanup the parallel psums example a little externals cleanup External cleanup fix Magic menu in qtconsole split in groups Minor bind_kernel improvements kernmagic during console startup Prevent jumping of window to input when output is clicked Fix 1px jumping of cells and menus in Notebook task fails with AssertionError not enough buffers after second resubmit fix chained resubmissions Rmagic extension Fix jumpy notebook behavior task with UnmetDependency error still owned by engine 2 5 Issues closed in the 0 13 development cycle 109 IPython Documentation Release 1 2 1 An Afternoon Hack 1847 1846 1843 1837 1834
409. nel Model example notebook 1 4 Interactive parallel computing Increasingly parallel computer hardware such as multicore CPUs clusters and supercomputers is becom ing ubiquitous Over the last several years we have developed an architecture within Python that allows such hardware to be used quickly and easily from Python Moreover this architecture is designed to support interactive and collaborative parallel computing The main features of this system are e Quickly parallelize Python code from an interactive Python IPython session e A flexible and dynamic process model that be deployed on anything from multicore workstations to supercomputers An architecture that supports many different styles of parallelism from message passing to task farm ing And all of these styles can be handled interactively Both blocking and fully asynchronous interfaces High level APIs that enable many things to be parallelized in a few lines of code Write parallel code that will run unchanged on everything from multicore workstations to supercom puters Full integration with Message Passing libraries MPI 4 Chapter 1 Introduction IPython Documentation Release 1 2 1 An Afternoon Hack e Capabilities based security model with full encryption of network connections e Share live parallel jobs with other users securely We call this collaborative parallel computing e Dynamically load balanced task farming system e Robust error
410. ng Controller Kernel restarting after message Kernel XX XX failed to respond to heartbeat cython magic gives ImportError dlopen long_file_name so 2 image not found In Python for Python 3 system site packages comes before user site packages Add magic watch function example Problem rendering pandas dataframe in Firefox for Windows notebook Figure legend not shown in inline backend if ouside the box of the axes notebook in Markdown mode press Enter key at the end of lt some http link gt the next line is indented unexpectly 3069 3444 3335 3386 API 3543 3549 3445 3237 2916 2167 3545 3534 3489 3396 3261 2984 3524 Instructions for installing Python notebook on Windows Encoding problem cannot use if user s name is not ascii Reenable bracket matching Magic paste not working in Python 3 3 2 TypeError Type str doesn t support the buffer Exception shutting down kernel from notebook dashboard 0 13 1 Codecell size changes with selection Adding newlines in latex cell notebook Can t close a notebook without errors colon invokes auto un indent in markdown cells Indent and dedent in htmInotebook Notebook save button icon not clear nbconvert incompatible with Windows Update example notebook that raw_input is allowed Notebook checkpoint time is displayed an hour out Empty revert to checkpoint menu if no checkpoint print
411. ng configuration and runtime files such as logs connection info for the parallel apps and your IPython command history The idea is that users often want to maintain a set of configuration files for different purposes one for doing numerical computing with NumPy and SciPy and another for doing symbolic computing with SymPy Profiles make it easy to keep a separate configuration files logs and histories for each of these purposes Let s start by showing how a profile is used ipython profile sympy This tells the ipython command line program to get its configuration from the sympy profile The file names for various profiles do not change The only difference is that profiles are named in a special way In the case above the sympy profile means looking for ipython_config py in lt IPYTHONDIR gt profile_sympy The general pattern is this simply create a new profile with S ipython profile create lt name gt which adds a directory called profile_ lt name gt to your Python directory Then you can load this profile by adding profile lt name gt to your command line options Profiles are supported by all Python applications Python ships with some sample profiles in I Python config profile If you create profiles with the name of one of our shipped profiles these config files will be copied over instead of starting with the automatically generated config files Security Files If you
412. ng for one Mark test_push_numpy_nocopy as a known failure for Python 3 fix tooltip on token with number pythonw py3k fixes for issue 1226 Add script to test pull request deprecate IPYTHON_DIR in favor of IPYTHONDIR Avoid deprecated warnings from ipython qtconsole desktop Add quote to notebook to allow it to load Fix sys path missing as first entry in ipython kernel import Binary from bson instead of pymongo Make IPython core display Image less notebook centric CLN Remove redundant function definition Point pastebin to gist handle pyout messages in test_message_spec add binary tree engine interconnect example Cherry picked commits from 0 12 1 release Handle carriage return characters r in HTML notebook output ensure kernels are cleaned up in embed_kernel tests InteractiveShell run_code Update docstring Delay flushing softspace until after cell finishes handle jpg jpeg in the qtconsole add patch_pyzmq for backporting a few changes from newer pyzmq DOC moving files with SSH launchers add IPython embed_kernel Finish up embed_kernel Remove bundled Itp module incremental improvements to SSH launchers move examples test_embed into examples tests embed Fix installing extension from local file on Windows 2 5 Issues closed in the 0 13 development cycle 97 IPython Documentation Release 1 2 1 An Afternoon Hack PR 1645 PR 1637 PR 1631 PR 1629 PR 1621 PR 1627 PR 1624 PR 1
413. ng pid file blue domainusers bgranger ipython profijle_mycluste PClusterStart riting job description file blue domainusers bgranger ipython profi PClusterStart tarting Win HPC Job job submit jobfile blue domainusers S bgranger i PClusterStart tarting 15 engines PClusterStart riting job description file blue domainusers bgranger ipython profi PClusterStart tarting Win HPC Job job submit jobfile blue domainusers bgranger i HHHHHHHH NEnNNnNBZAN At this point the controller and two engines are running on your local host This configuration is useful for testing and for situations where you want to take advantage of multiple cores on your local computer Now that we have confirmed that ipcluster is working properly we describe how to configure and run an Python cluster on an actual compute cluster running Windows HPC Server 2008 Here is an outline of the needed steps 1 Create a cluster profile using ipython profile create mycluster parallel 2 Edit configuration files in the directory ipython cluster_mycluster 3 Start the cluster using ipcluster start profile mycluster n 32 Creating a cluster profile In most cases you will have to create a cluster profile to use Python on a cluster A cluster profile is a name like mycluster that is associated with a particular cluster configuration The profile name is used by ipcluster when working with th
414. ngines If an attacker were able to connect a hostile engine to a user s controller the user might unknowingly send sensitive code or data to the hostile engine This attacker s engine would then have full access to that code and data This type of attack is prevented in the same way as the unauthorized client attack through the usage of the capabilities based authentication scheme 5 9 Security details of IPython 313 IPython Documentation Release 1 2 1 An Afternoon Hack Unauthorized controllers It is also possible that an attacker could try to convince a user s Python client or engine to connect to a hostile Python controller That controller would then have full access to the code and data sent between the Python client and the Python engines Again this attack is prevented through the capabilities in a connection file which ensure that a client or engine connects to the correct controller It is also important to note that the connection files also encode the IP address and port that the controller is listening on so there is little chance of mistakenly connecting to a controller running on a different IP address and port When starting an engine or client a user must specify the key to use for that connection Thus in order to introduce a hostile controller the attacker must convince the user to use the key associated with the hostile controller As long as a user is diligent in only using keys from trusted sourc
415. nnecting the heartbeat process and kernel queue socket s The message status will be an error if the Engine requests IDs that already in use Message type registration_reply 402 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack content status ok or error if ok id 0 int the engine id Clients use the same socket as engines to start their connections Connection requests from clients need no information Message type connection_request content The reply to a Client registration request contains the connection information for the multiplexer and load balanced queues as well as the address for direct hub queries If any of these addresses is None that functionality is not available Message type connection_reply content status ok or error Heartbeat The hub uses a heartbeat system to monitor engines and track when they become unresponsive As de scribed in messaging and shown in connections Notification PUB The hub publishes all engine registration unregistration events on a PUB socket This allows clients to have up to date engine ID sets without polling Registration notifications contain both the integer engine ID and the queue ID which is necessary for sending messages via the Multiplexer Queue and Control Queues Message type registration_notification co
416. nnot restart Notebook when using script bg Clean BG processes created by script on kernel exit Fix RST misformatting Introduce Notebook Magics Kernels should start in notebook dir when manually specified Notebook should check that notebook dir exists minor notebook startup notebook dir adjustments tab completion in notebook for paths not triggered Allow path completion on notebook allow multiple instances of a Magic page not working fix _ofind attr in page Shutdown qtconsole problem check for active frontend in update_restart_checkbox Add support for tox http tox testrun org and Travis CI http travis ci org Parallel output of px and px suffix is inconsistent ValueError packer could not serialize a simple message Notebook MathJax offline install not recognized dblclick to restore size of images Notebook does not save heading level Notebook names truncating at the first period Limited size of output cells and provide scroll bars for such output cells second attempt at scrolled long output add cell level metadata Cell Worksheet metadata Confirm restart configuration option and checkbox UI Commenting function Tab completion problems with cell magics 2 5 Issues closed in the 0 13 development cycle 105 IPython Documentation Release 1 2 1 An Afternoon Hack 1944 1973 1972 1887 1967 1678 1965 1952 1961 1963 1962 1959 1955 1945 1957 1956
417. not PageUp more than once Store exit status of last command Users should be able to open different notebooks in the cwd Users should see a consistent behavior in the Out prompt in the html notebook Notebook should not import anything by default 2 9 Issues closed in the 0 11 development cycle 173 IPython Documentation Release 1 2 1 An Afternoon Hack 595 588 586 585 584 e 580 569 e 572 e 573 e 570 e 566 e 562 558 e 554 e 553 549 e 505 e 545 371 e 377 e 544 e 543 e 440 e 541 e 369 351 e 512 e 340 qtconsole command issue ipython qtconsole uses 100 CPU plot Command BOrks QTConsole Strangely pdoc throws Errors for classes without __init__ or docstring pdoc throws TypeError Client instantiation AssertionError UnicodeDecodeError during startup Indented command hits error wthread breaks indented top level statements pylab inline vs pylab inline Can t use exec_file in config file update docs to reflect args values triple quote and s at beginning of line Update 0 11 docs to explain Qt console and how to do a clean install embed fails if config files not installed 8 Ensure gui qt works with new Mayavi and pylab 269 Provide compatibility api for IPython Shell start mainloopQ 66 Update the main What s New document to reflect work on 0 11 Don t check for linux2 value in sys pl
418. notebook fixups for alternate URL prefix stuff crack at making notebook html use the layout html template RST and heading cells 122 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e 1350 1247 873 1139 1329 1332 1317 1341 1300 1334 1338 1335 1299 1318 1314 1328 1206 1330 1309 1327 1326 1325 1316 1315 1233 1234 1235 1236 1303 1311 1304 1312 Add m mod run library module as a script option fixes a bug causing extra newlines after comments add base_url to notebook configuration options notebook allow prefixes in URL path Very slow traceback construction from Cython extension Don t attempt to tokenize binary files for tracebacks Cell Input collapse added key handler for control s to notebook seems to work pretty well Fix see also in docstrings so API docs build Notebook toolbar UI made notebook html extend layout html make Ctrl D in gtconsole act same as in terminal ready to merge ReST support in notebook frontend Notebook webkit notification Insertcell Coverage don t preserve fixConsole output in json Add linewrapping to text cells new feature in CodeMirror Inoculate clearcmd extension into reset functionality Updatecm2 Removing Ace edit capability forgotten selected_cell gt get_selected_cell Pass subprocess test runners a suitable location f
419. nt error message for run nonexistent_file 651 Completion doesn t work on element of a list 617 qtconsole hist doesn t show anything in qtconsole 786 from __future__ import unicode_literals does not work 779 Using irunner from virtual evn uses systemwide ipython 768 codepage handling of output from scripts and shellcommands are not handled properly by qt console 785 Don t strip leading whitespace in repr in notebook 737 in pickleshare py line52 should be if not os path isdir self root 738 in ipthon_win_post_install py line 38 TTT print sep raises SyntaxError 728 ipcontroller crash with MPI 780 qtconsole Out value prints before the print statements that precede it 632 Python Crash Report 0 10 2 253 Unable to install ipython on windows 80 Split IPClusterApp into multiple Application subclasses for each subcommand 34 non blocking pendingResult partial results 739 Tests fail if tornado not installed 719 Better support Pypy 667 qtconsole problem with default pylab profile 661 ipythonre referenced in magic command in 0 11 665 Source introspection with is broken 724 crash ipython qtconsole Z quickref 2 7 Issues closed in the 0 12 development cycle 151 IPython Documentation Release 1 2 1 An Afternoon Hack 655 593 563 dol 720 740 T12 TIA 767 T33 721 669 J11 500 707 478 694 704 442 615 614 606 604 298 649
420. nt to the Python controller they could instruct the engines to execute code On the first level this attack is prevented by requiring access to the controller s ports which are recom mended to only be open on loopback if the controller is on an untrusted local network If the attacker does have access to the Controller s ports then the attack is prevented by the capabilities based client authentica tion of the execution key The relevant authentication information is encoded into the JSON file that clients must present to gain access to the Python controller By limiting the distribution of those keys a user can grant access to only authorized persons just as with SSH keys It is highly unlikely that an execution key could be guessed by an attacker in a brute force guessing attack A given instance of the Python controller only runs for a relatively short amount of time on the order of hours Thus an attacker would have only a limited amount of time to test a search space of size 2 128 For added security users can have arbitrarily long keys Warning If the attacker has gained enough access to intercept loopback connections on either the controller or client then a duplicate message can be sent To protect against this recipients only allow each signature once and consider duplicates invalid However the duplicate message could be sent to another recipient using the same key and it would be considered valid Unauthorized e
421. ntent id 0 engine ID that has been registered uuid engine_id the IDENT for the engine s sockets Message type unregistration_notification content id 0 engine ID that has been unregistered uuid engine_id the IDENT for the engine s sockets 7 3 Messaging for Parallel Computing 403 IPython Documentation Release 1 2 1 An Afternoon Hack Client Queries ROUTER The hub monitors and logs all queue traffic so that clients can retrieve past results or monitor pending tasks This information may reside in memory on the Hub or on disk in a database SQLite and MongoDB are currently supported These requests are handled by the same socket as registration queue_request requests can specify multiple engines to query via the targets element A verbose flag can be passed to determine whether the result should be the list of msg_ids in the queue or simply the length of each list Message type queue_request content verbose True whether return should be lists themselves or just lens targets 0 3 1 list of ints The content of a reply to a queue_request request is a dict keyed by the engine IDs Note that they will be the string representation of the integer keys since JSON cannot handle number keys The three keys of each dict are completed messages submitted via any queue that ran on the engine queue jobs
422. nterpreter x x x exit_msg x Back in main IPython xxx First import the embed function from IPython terminal embed import InteractiveShellEmbed Now create the IPython shell instance Put ipshell anywhere in your code where you want it to open ipshell InteractiveShellEmbed bannerl banner exit_msg exit_msg This code will load an embeddable IPython shell always with no changes for nested embededings from IPython import embed Now embed will open IPython anywhere in the code This code loads an embeddable shell only if NOT running inside IPython Inside IPython th mbeddable shell variable ipshell is just a dummy function try get_ipython except NameError from IPython terminal embed import InteractiveShellEmbed ipshell InteractiveShellEmbed 224 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack Now ipshell will open IPython anywhere in the code else Define a dummy ipshell so the same code doesn t crash inside an interactive IPython def ipshell pass Pee KKKK KKK KK KKK End Of file lt example embed Short Py gt KxKKKKKKKKKKK KKK KK AK 4 3 5 Using the Python debugger pdb Running entire programs via pdb pdb the Python debugger is a powerful interactive debugger which allows you to step through code set breakpoints watch variables etc Python makes
423. nters Clear window title when kernel is restarted Fix for bug 735 Images missing from XML SVG export Tooltip completer js refactor add qt config option to clear_on_kernel_restart Tooltip completer js refactor terminate kernel after embed_kernel tests add HistoryManager to ipapp class list ENH Open a notebook from the command line fixes 1708 failing test in arg_split on windows Use CRegExp trait for regular expressions Catch failure in repr for whos use eval for command line args instead of exec fix scatter gather with targets all add no ff to git pull in test_pr Tooltip completer js refactor Add wait optional argument to hooks editor Define generic sys ps 1 2 3 for use by scripts PR 1691 PR 1710 Finish PR 1446 update MathJax CDN url for https 96 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 1713 PR 1703 PR 1714 PR 1704 PR 1706 PR 1698 PR 1245 PR 1685 PR 1693 PR 1695 PR 1694 PR 1689 PR 1687 PR 1616 PR 1684 PR 1670 PR 1669 PR 1295 PR 1642 PR 1659 PR 1656 PR 1664 PR 1662 PR 1643 PR 1652 PR 1650 PR 1357 PR 1640 PR 1651 PR 1634 PR 1649 PR 1633 Make autocall regexp s configurable Allow TryNext to have an error message without it affecting the command chain minor adjustments to test_pr ensure all needed qt parts can be imported before settli
424. ntext SyntaxError display broken in Python 3 Fix display of SyntaxError in Python 3 Dewijmoize Skip tests that require X when importing pylab results in RuntimeError Wijmoize can not imput chinese word exit right now Adding Opera 11 as a compatible browser for ipython notebook Kernel Has Died error in Notebook serve local files in notebook dir edit text cells on double click instead of single click misc notebook connection file cleanup first heartbeat startup flush fix loadpy duplicating newlines Always save the py file to disk next to the ipynb execute cell in place should preserve the current insertion point in the notebook In numbers are not invalidated when restarting kernel pip on OSX tries to install files in System directory Unified setup py Reduce Python external Make all the static files path absolute Added q option to prun for suppression of the output along with editing the dochelp string Added q option to prun for suppression of the output along with editing the dochelp string Pdb tab completion does not work in QtConsole Interactive shell trying to from import history core completer Clean up excessive and unused code should dv sync_import print failed imports payloadpage py not used by qtconsole 2 5 Issues closed in the 0 13 development cycle 125 IPython Documentation Release 1 2 1 An Afternoon Hack 1204 double newline from loadpy in python notebook at
425. o svg file fixes 489 add QtConsoleApp using newapplication embed doesn t load default config Links launchpad gt github xdel magic Add n to lines in the log use os system for shell system in Terminal frontend 1473 Add test decorator onlyif_unicode_paths Fix support for raw GTK and WX matplotlib backends Kernel event loop is robust against random SIGINT Share code for magic_edit Add exit code when running all tests with iptest Add home directory expansion to IPYTHON_DIR environment variables Bugfix with logger Separate out skip_doctest decorator Draft of new main BaseIPythonApplication Use list tuple dict set subclass s overridden __repr__ instead of the pretty allow toggle of svg png inline figure format Support inline PNGs of matplotlib plots Retries and Resubmit 411 and 412 Fixes to the display system 168 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e 449 Fix issue 447 inspecting old style classes e 423 Allow type checking on elements of List Tuple Set traits e 400 Config5 e 421 Generalise mechanism to put text at the next prompt in the Qt console e 443 pinfo code duplication e 429 add check_pid and handle stale PID info in ipcluster e 431 Fix error message in test_irunner e 427 handle different SyntaxError messages in test_irunner e 424 Irunner test failure e 430 Small parallel doc typo e 422 Make ipython qtc
426. o apply has the following signature view apply f xargs xkwargs There are various ways to call functions with Python and these flags are set as attributes of the View The Direct View has just two of these flags dv block bool whether to wait for the result or return an AsyncResult object immediately dv track bool whether to instruct pyzmq to track when zeromq is done sending the message This is primarily useful for non copying sends of numpy arrays that you plan to edit in place You need to know when it becomes safe to edit the buffer without corrupting the message dyv targets int list of ints which targets this view is associated with Creating a view is simple index access on a client creates a DirectView In 4 view re 1 3 Out 4 lt DirectView 1 2 gt In 5 view apply lt tab gt view apply view apply_async view apply_sync For convenience you can set block temporarily for a single call with the extra sync async methods Blocking execution In blocking mode the Di rect View object called dview in these examples submits the command to the controller which places the command in the engines queues for execution The apply call then blocks until the engines are done executing the command In 2 dview rce A DirectView of all engines In 3 dview block True In 4 dview a 5 In 5 dview b 10 5 3 IPython s Direct interface 281
427. o run a file as a demo you must first make a Demo object out of it If the file is named myscript py the following code will make a demo 4 3 IPython reference 229 IPython Documentation Release 1 2 1 An Afternoon Hack from IPython 1lib demo import Demo mydemo Demo myscript py This creates the mydemo object whose blocks you run one at a time by simply calling the object with no arguments If you have autocall active in Python the default all you need to do is type mydemo and Python will call it executing each block Demo objects can be restarted you can move forward or back skipping blocks re execute the last block etc Simply use the Tab key on a demo object to see its methods and call on them to see their docstrings for more usage details In addition the demo module itself contains a comprehensive docstring which you can access via from IPython lib import demo demo Limitations It is important to note that these demos are limited to fairly simple uses In particular you cannot break up sections within indented code loops if statements function definitions etc Supporting something like this would basically require tracking the internal execution state of the Python interpreter so only top level divisions are allowed If you want to be able to open an Python instance at an arbitrary point in a program you can use Python s embedding facilities see Python
428. ocumentation Release 1 2 1 An Afternoon Hack MongoDB queries so if you have used MongoDB it should look familiar In fact when the MongoDB backend is in use the query is relayed directly When using other backends the interface is emulated and only a subset of queries is possible See also MongoDB query docs http www mongodb org display DOCS Querying Client db_query takes a dictionary query object with keys from the TaskRecord key list and val ues of either exact values to test or MongoDB queries which are dicts of The form operator argument s There is also an optional keys argument that specifies which subset of keys should be retrieved The default is to retrieve all keys excluding the request and result buffers db_query returns a list of TaskRecord dicts Also like MongoDB the msg_id key will always be included whether requested or not TaskRecord keys Key Type Description msg_id uuid ascii The msg ID header dict The request header content dict The request content likely empty buffers list bytes buffers containing serialized request objects submitted datetime timestamp for time of submission set by client client_uuid uuid ascii IDENT of client s socket engine_uuid uuid ascii IDENT of engine s socket started datetime time task began execution on engine completed datetime time task finished execution success or f
429. odel suggests that execution be able to use the model ns apply f xargs xkwargs which takes f a function in the user s namespace and executes f xargs x kwargs on a remote engine returning the result or for non blocking information facilitating later retrieval of the result This model unlike the execute message which just uses a code string must be able to send arbitrary pickleable Python objects And ideally copy as little data as we can The buffers property of a Message was introduced for this purpose Utility method build_apply_message in IPython kernel zmq serialize wraps a func tion signature and builds a sendable buffer format for minimal data copying exactly zero copies of numpy array data or buffers or large strings Message type apply_request metadata after msg_id list of msg_ids or output of Dependency as_dict follow msg_id list of msg_ids or output of Dependency as_didat content buffers at least 3 in length as built by build_apply_message f args kwargs after follow represent task dependencies after corresponds to a time dependency The request will not arrive at an engine until the after dependency tasks have completed follow corresponds to a location dependency The task will be submitted to the same engine as these msg_ids see Dependency docs for details Message type apply_reply
430. ollerApp IPControllerApp TPControllerApp Sc ipcontroller ip 192 168 1 16 Hub listening on tcp 192 168 1 16 63320 for registrati hub created hub writing connection info to Users me ipython profile_de writing connection info to Users me ipython profile_de task using Python leastload Task scheduler Heartmonitor started Using existing profile dir u Users me ipython profile default pn Hub using DB backend IPython parallel controller dictdb DictDB Fault securi Fault securi Creating pid file Users me ipython profile_default pid ipcontroll heduler started leastload 2 on each engine fetch the connection file with scp engine hos t n scp controller host ipython profile_default security ip controller e Note The log output of ipcontroller above shows you where the json files They will be in ipython or config ipython under profile_default security ipcontroller engine json were written 3 start the engines using the connection file engine host n ipengin file ipcontroller engine json A couple of notes e You can avoid having to fetch the connection file every time by adding reuse flag to ipcontroller which instructs the controller to read the previous connection file for connection info rather than generate a new one with randomized ports e In step 2 if you fetch the connection file directly into the
431. om Fetherston lt tfetherston at aol com gt Tom MacWright tzanko vankayala sowjanya lt hai sowjanya at gmail com gt Vivian De Smedt lt vds2212 at VIVIAN gt Ville M Vainio lt vivainio at gmail com gt Vishal Vatsa lt vishal vatsa at gmail com gt Vishnu S G lt sgvishnu777 at gmail com gt 2 8 0 11 Series 165 IPython Documentation Release 1 2 1 An Afternoon Hack e Walter Doerwald lt walter at livinglogic de gt Note This list was generated with the output of git log dev 0 11 HEAD format aN lt SaE gt sed s at sed s lt gt sort u after some cleanup If you should be on this list please add yourself 2 9 Issues closed in the 0 11 development cycle In this cycle we closed a total of 511 issues 226 pull requests and 285 regular issues this is the full list generated with the script tools github_stats py We should note that a few of these were made on the 0 10 x series but we have no automatic way of filtering the issues by branch so this reflects all of our development over the last two years including work already released in 0 10 2 Pull requests 226 e 620 Release notes and updates to GUI support docs for 0 11 e 642 fix typo in docs examples vim README rst e 631 two way vim ipython integration e 637 print is a function this allows to properly exit ipython e 635 support html representations in the notebook frontend e 639 Updating the cr
432. omize the arguments passed to the Python instance at startup by setting the py python command args variable For example to start always with matplot1ib integration and hardcoded light background colors you can use setq py python command args matplotlib colors LightBG If you happen to get garbage instead of colored prompts as described in the previous section you may need to set also in your emacs file 362 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack setq ansi color for comint mode t Notes on emacs support e There is one caveat you should be aware of you must start the IPython shell before attempting to execute any code regions via C c Simply type C c to start Python before passing any code regions to the interpreter and you shouldn t experience any problems This is due to a bug in Python itself which has been fixed for Python 2 3 but exists as of Python 2 2 2 reported as SF bug 737947 D The X Emacs support is maintained by Alexander Schmolck so all comments requests should be directed to him through the IPython mailing lists This code is still somewhat experimental so it s a bit rough around the edges although in practice it works quite well Be aware that if you customized py python command previously this value will override what ipython el does because loading the customization variables co
433. on of Views 5 5 3 Quick and easy parallelism In many cases you simply want to apply a Python function to a sequence of objects but in parallel Like the multiengine interface these can be implemented via the task interface The exact same tools can per form these actions in load balanced ways as well as multiplexed ways a parallel version of map and parallel function decorator If one specifies the argument balanced True then they are dynami cally load balanced Thus if the execution time per item varies significantly you should use the versions in the task interface Parallel map To load balance map simply use a LoadBalanced View In 62 lview block True In 63 serial_result map lambda x xx x10 range 32 In 64 parallel_result lview map lambda x xx 10 range 32 296 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack In 65 serial_result parallel_result Out 65 True Parallel function decorator Parallel functions are just like normal function but they can be called on sequences and in parallel The multiengine interface provides a decorator that turns any Python function into a parallel function In 10 lview parallel sansi def f x arrat return 10 0xxx 4 In 11 f map range 32 this is done in parallel Out 11 0 0 10 0 160 0 5 5 4 Dependencies Often
434. on process that is used to coordinate the engines to get a parallel computation done Collectively these processes are called the Python cluster and the hub and schedulers together are referred to as the controller These processes communicate over any transport supported by ZeroMQ tcp pgm infiniband ipc with a well defined topology The Python hub and schedulers listen on sockets Upon starting an engine connects to a hub and registers itself which then informs the engine of the connection information for the schedulers and the engine then connects to the schedulers These engine hub and engine scheduler connections persist for the lifetime of each engine The Python client also connects to the controller processes using a number of socket connections As of writing this is one socket per scheduler 4 and 3 connections to the hub for a total of 7 These connections persist for the lifetime of the client only A given Python controller and set of engines engines typically has a relatively short lifetime Typically this lifetime corresponds to the duration of a single parallel simulation performed by a single user Finally the hub schedulers engines and client processes typically execute with the permissions of that same user More specifically the controller and engines are not executed as root or with any other superuser permissions 5 9 2 Application logic When running the Python kernel to perform a parallel computation a us
435. on related magic functions qtconsole does not accept gui switch Accept gui lt gt switch in Python qtconsole Fix comment which breaks Emacs syntax highlighting gui magic broken fix Ygui magic extreme truncating of return values Raise repr limit for strings to 80 characters from 30 don t use XDG path on OS X ipython crash on wrong encoding Unicode aware logger update zmqshell magics DOC Remove regression from qt console docs IPython utils tests test_process SubProcessTestCase 2 5 Issues closed in the 0 13 development cycle 111 IPython Documentation Release 1 2 1 An Afternoon Hack 1758 1786 1748 1755 1771 1783 1784 1768 1778 1779 1769 1716 1763 1766 1611 1732 1765 1630 1705 1606 1720 1701 1757 1735 1754 1753 1751 1749 1743 test_pr fallback on http if git protocol fail and SSL errors Make notebook save failures more salient Fix some tests for Python 3 3 test for pygments before running qt tests Make default value of interactivity passed to run_ast_nodes configurable part of PR 1606 loadpy gt load erased by magic refactoring restore loadpy to load Update parallel magics string exception in Python core magic py 232 Tidy up error raising in magic decorators Allow cell mode timeit without setup code Fix for fake filenames in verbose traceback qtconsole fix append_plain_html gt append_htm
436. on3 Add magic for cls on windows Fix for 181 enable gui pylab magics in the Kernel Allow Python to run without sqlite3 Qtconsole menu notebook download implies save 140 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 896 899 892 893 789 898 897 877 838 872 Execfile Brian s Notebook work don t close figures every cycle with inline matplotlib backend Adding clear_output to kernel and HTML notebook Adding clear_output to kernel and HTML notebook Don t pass unicode sys argv with run or ipython script py Add tooltips to the notebook via title attr partial fix for issue 678 reenable multiline history for terminals The constructor of Client checks for AssertionError in validate_url to open a file instead of connection to a URL if it fails 884 883 889 891 813 876 856 868 878 824 850 861 819 855 862 847 708 857 835 859 848 Notebook usability fixes User notification if notebook saving fails Add drop_by_id method to shell to remove variables added by extensions Ability to open the notebook in a browser when it starts Create menu bar for qtconsole protect Python from bad custom exception handlers Backgroundjobs Warn user if MathJax can t be fetched from notebook closes 744 store_history False default for run_cell History access Update codemirror to 2 15 and make the code internal
437. onfigurable objects implement the actual logic of the application and know how to configure themselves given the configuration object Applications always have a log attribute that is a configured Logger This allows centralized logging configuration per application Configurable Configurable A configurable is a regular Python class that serves as a base class for all main classes in an application The Configurable base class is lightweight and only does one things This Configurable is a subclass of HasTraits that knows how to configure itself Class level traits with the metadata config True become values that can be configured from the command line and configuration files 347 IPython Documentation Release 1 2 1 An Afternoon Hack Developers create Configurable subclasses that implement all of the logic in the application Each of these subclasses has its own configuration information that controls how instances are created Singletons SingletonConfigurable Any object for which there is a single canonical instance These are just like Configurables except they have a class method instance that returns the current active instance or creates one if it does not exist Examples of singletons include InteractiveShell This lets objects easily connect to the current running Application with out passing objects around everywhere For instance to get the current running Application instance simply do app Application instan
438. onsole a GUI script e 420 Permit kernel std to be redirected e 408 History request e 388 Add Emacs style kill ring to Qt console e 414 Warn on old config files e 415 Prevent prefilter from crashing Python e 418 Minor configuration doc fixes e 407 Update What s new documentation e 410 Install notebook frontend e 406 install Python zmq gui e 393 ipdir unicode e 397 utils io Term cin out err gt utils io stdin out err e 389 DB fixes and Scheduler HWM e 374 Various Windows related fixes to Python parallel e 362 fallback on defaultencoding if filesystemencoding is None e 382 Shell s reset method clears namespace from last run command e 385 Update iptest exclusions fix 375 e 383 Catch errors in querying readline which occur with pyreadline e 373 Remove runlines etc e 364 Single output e 372 Multiline input push e 363 Issue 125 2 9 Issues closed in the 0 11 development cycle 169 IPython Documentation Release 1 2 1 An Afternoon Hack 361 349 359 356 330 309 don t rely on setuptools for readline dependency check Fix Yautopx magic History save thread Usability improvements to history in Qt console Exit autocall Rewrite quit exit Quit ExitQ calls as magic Cell tweaks Attempt to address partly issue ipython 342 by rewriting quit exitQ etc 342 Try to recover as intelligently as possible if user calls magic Dedent prefix bugfix
439. ook server Naming releases TypeError must be string without null bytes not str at end of comment causes line to execute nbconvert add logging for the different steps of nbconvert Markdown cells do not render correctly when mathjax is disabled nbconvert to rst leftover In J nbconvert Ability to specify name of output file Document when IPython start_ipython should be used versus Python embed Add no more readonly view in what s new No Print View in Notebook in 1 0dev Python 0 12 1 Crashes on autocompleting sqlalchemy func row_number properties Opening notebook directly from the command line with multi directory support installed Annoying behavior when clicking on cell after execution Ctrl Enter Possible to add some bpython features Printing the contents of an image file messes up shell text nbconvert Default help message should be that of help Nbconvert 1 0 0a1 does not take into account the pdf extensions in graphs 50 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 3719 3786 3019 3785 3796 3788 1366 3739 3768 3724 1592 2049 3757 3745 3701 3738 3740 3737 3730 3718 3710 3713 3712 3680 3698 3693 3688 3542 2556 3684 3682 3596 Bad strftime format for windows in nbconvert exporter Zmq errors appearing with Ctrl C in console qtconsole disappearing scroll
440. ook very similar to an Python config file Python applications use a parser called KeyValueLoader to load values into a Config object Values are assigned in much the same way as in a config file ipython InteractiveShell use_readline Fals BaseIPythonApplication profil Is the same as adding c InteractiveShell use_readline Fals c BaselPythonApplication profile myprofile to your config file Key Value arguments always take a value separated by and no spaces Common Arguments Since the strictness and verbosity of the KVLoader above are not ideal for everyday use common arguments can be specified as flags or aliases Flags and Aliases are handled by argparse instead allowing for more flexible parsing In general flags and aliases are prefixed by except for those that are single characters in which case they can be specified with a single e g ipython i c import numpy x numpy linspace 0 1 profile testing colon Aliases For convenience applications have a mapping of commonly used traits so you don t have to specify the whole class name ipython profile myprofile and ipython profile myprofile are equivalent to ipython BaseIPythonApplication profile myprofile Ws OW WD 6 1 Overview of the IPython configuration system 353 e myprofils s lightbg IPython Documentation Release 1 2 1 An Afternoon
441. oon Hack 2 10 3 Release 0 10 This release brings months of slow but steady development and will be the last before a major restructuring and cleanup of Python s internals that is already under way For this reason we hope that 0 10 will be a stable and robust release so that while users adapt to some of the API changes that will come with the refactoring that will become Python 0 11 they can safely use 0 10 in all existing projects with minimal changes if any Python 0 10 is now a medium sized project with roughly as reported by David Wheeler s sloccount utility 40750 lines of Python code and a diff between 0 9 1 and this release that contains almost 28000 lines of code and documentation Our documentation in PDF format is a 495 page long PDF document also available in HTML format both generated from the same sources Many users and developers contributed code features bug reports and ideas to this release Please do not hesitate in contacting us if we ve failed to acknowledge your contribution here In particular for this release we have contribution from the following people a mix of new and regular names in alphabetical order by first name Alexander Clausen fix 341726 Brian Granger lots of work everywhere features bug fixes etc Daniel Ashbrook bug report on MemoryError during compilation now fixed Darren Dale improvements to documentation build system feedback design ideas Fernando Perez
442. opx The autopx magic switches to a mode where everything you type is executed on the engines until you do autopx again H SE evals numpy linalg eigvals a PENE max_evals append evals n 31 autopx Sautopx enabled In 30 dv block True In 32 max_evals In 33 for i in range 100 numpy random rand 1l ata transpose 0 10 0 real o In 34 print Average max eigenvalue is f sum max_evals len max_evals stdout 0 Average max eigenvalue is 10 193101 stdout 1 Average max eigenvalue is 10 064508 stdout 2 Average max eigenvalue is 10 055724 stdout 3 Average max eigenvalue is 10 086876 292 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack In 35 Sautopx Auto Parallel Disabled pxconfig The default targets and blocking behavior for the magics are governed by the block and targets attribute of the active View If you have a handle for the view you can set these attributes directly but if you don t you can change them with the pxconfig magic In 3 pxconfig block In 5 px print hi Parallel execution on engine s all stdout 0 hi stdout 1 hi stdout 2 hi stdout 3 hi In 6 pxconfig targets 2 In 7 px print hi Parallel execution on engine s 0 2 stdout 0 hi stdout 2 hi In 8 Spxconfig noblock
443. or word gives access to the full information including the source code where possible Long strings are not snipped The following magic functions are particularly useful for gathering information about your working environ ment You can get more details by typing smagic or querying them individually sfunction_name this is just a summary pdoc lt object gt Print or run through a pager if too long the docstring for an object If the given object is a class it will print both the class and the constructor docstrings pdef lt object gt Print the call signature for any callable object If the object is a class print the constructor information psource lt object gt Print or run through a pager if too long the source code for an object pfile lt object gt Show the entire source file where an object was defined via a pager opening it at the line where the object definition begins who whos These functions give information about identifiers you have defined interactively not things you loaded or defined in your configuration files who just prints a list of identifiers and Yowhos prints a table with some basic details about each identifier Note that the dynamic object information functions Spdoc spfile spdef spsource work on object attributes as well as directly on variables For example after doing import os you can use os path abspath Readline based features These features require the
444. or xunit output Collect results from subprocess runners and spit out Xunit XML output Update CodeMirror to the latest version Refactor how the notebook focuses cells After upgrading CodeMirror check the status of some bugs Review how select is called when notebook cells are inserted Updatecm Fixing CM related indentation problems controller server load can disrupt heartbeat minor heartbeat tweaks 2 5 Issues closed in the 0 13 development cycle 123 IPython Documentation Release 1 2 1 An Afternoon Hack 1302 1306 1251 1202 Input parsing with prun clobbers escapes Fix Yoprun input parsing for escaped characters closes 1302 IPython 0 12 can t import map module on Python 3 1 Pyreadline install exclusion for 64 bit windows no longer required version dependency not correctly specified 1301 1242 1203 1289 1263 1259 1278 1210 1288 1290 1291 1293 1294 1281 1283 1162 1264 1261 1269 1284 1232 1279 1276 1213 1248 1267 1222 New Fix for issue 1202 based on current master changed key map name to match changes to python mode Fix for issue 1202 Make autoreload extension work on Python 3 Different C x for shortcut C m c not toCodeCell anymore Replace from I import with absolute imports took a crack at making notebook html extend layout html Add quiet option to suppress screen output during pru
445. ort for running inside a virtualenv While this doesn t supplant proper installation as users should do it helps ad hoc calling of Python from inside a virtualenv PR 1388 Major Bugs fixed In this cycle we have closed over 740 issues but a few major ones merit special mention e The Spastebin magic has been updated to point to gist github com since unfortunately http paste pocoo org has closed down We also added a d flag for the user to provide a gist de scription string PR 1670 Fix Spaste that would reject certain valid inputs PR 1258 Fix sending and receiving of Numpy structured arrays those with composite dtypes often used as recarrays PR 2034 Reconnect when the websocket connection closes unexpectedly PR 1577 Fix truncated representation of objects in the debugger by showing at least 80 characters worth of information PR 1793 Fix logger to be Unicode aware logging could crash ipython if there was unicode in the input PR 1792 Fix images missing from XML SVG export in the Qt console PR 1449 Fix deepreload on Python 3 PR 1625 as well as having a much cleaner and more robust implemen tation of deepreload in general PR 1457 Backwards incompatible changes e The exception IPython core error TryNext previously accepted arguments and keyword arguments to be passed to the next implementation of the hook This feature was removed as it made error message propagation difficult and viola
446. ory system SQLite history Unbundle external libraries Magic arguments Emacs completion bugfix Issue 133 Writing unicode characters to a log file Python 0 10 2 git Support for 256 color escape sequences in Qt console Refactored and improved Qt console s HTML export facility Fix185 take two Issue 129 add basic XDG_CONFIG_HOME support inline pylab cuts off labels on log plots Add precision magic Pyside support Make ipython cProfile able Magic examples Doc magic pycat Doc magic alias Doc magic edit Doc magic cpaste Doc magic pdef Docs build Doc magic who Doc magic cd 2 9 Issues closed in the 0 11 development cycle 171 IPython Documentation Release 1 2 1 An Afternoon Hack 226 218 231 225 222 223 203 227 232 264 220 268 267 266 255 265 260 243 242 250 239 244 234 217 215 195 192 198 174 188 158 187 Doc magic pwd Doc magic history Doc magic reset Doc magic save Doc magic timeit Doc magic colors Small typos in zmq blockingkernelmanager py Doc magic logon Doc magic profile Kernel logging Doc magic edit PyZMQ gt 2 0 10 GitHub Pages again OSX specific fixes to the Qt console Gitwash typo Fix string input2 Kernel crash with empty history New display system Fix terminal exit always use Session send Makefile command amp script for GitHub Pages My exit Timed history save Doc magic Ismagic History fix Fo
447. otInteractive py ipy_p4 py outputtrap py GnuplotRuntime py ipy_profile_none py PhysicalQInput py ibrowse py ipy_profile_numpy py PhysicalQInteractive py igrid py ipy_profile_scipy py quitter py ipipe py ipy_profile_sh py scitedirector py iplib py ipy_profile_zope py Shell py ipy_constants py ipy_traits_completer py twshell py Other regressions e The machinery that adds functionality to the sh profile for using Python as your system shell has not been updated to use the new APIs As a result only the aesthetic prompt changes are still implemented We intend to fix this by 0 12 Tracked as issue 547 e The installation of scripts on Windows was broken without setuptools so we now depend on se tuptools on Windows We hope to fix setuptools less installation and then remove the setuptools dependency Issue 539 e The directory history _dh is not saved between sessions Issue 634 Removed Features As part of the updating of IPython we have removed a few features for the purposes of cleaning up the code base and interfaces These removals are permanent but for any item listed below equivalent functionality is available e The magics Exit and Quit have been dropped as ways to exit Python Instead the lowercase forms of both work either as a bare name exit or a function call exit You can assign these to other names using exec_lines in the config file Credits Many users and developers contributed code features
448. ou can prepare a hashed password using the function IPython lib security passwd In 1 from IPython lib import passwd In 2 passwd Enter password Verify password Out 2 shal 67c9e60bb8b6 If Ffede0825894254b2e042ea597d771089ellaed Note passwd can also take the password as a string argument Do not pass it as an argument inside an Python session as it will be saved in your input history You can then add this to your ipython_notebook_config py e g Password to use for web authentication c get_config 4 8 Running a notebook server 255 IPython Documentation Release 1 2 1 An Afternoon Hack c NotebookApp password u shal 67c9e60bb8b6 If fed 0825894254b2e042ea597d771089ellaed When using a password it is a good idea to also use SSL so that your password is not sent unencrypted by your browser You can start the notebook to communicate via a secure protocol mode using a self signed certificate with the command ipython notebook cert file mycert pem Note A self signed certificate c an be generated with openssl For example the following command will create a certificate valid for 365 days with both the key and certificate data written to the same file openssl req x509 nod s days 365 newkey rsa 1024 keyout mycert pem out n Your browser will warn you of a dangerous certificat
449. out togit co The easiest way to do this is to create a git config file in your home directory with contents like this core editor emacs user email you yourdomain example com name Your Name Comes Here alias st status stat status co checkout color diff auto status true of course you ll need to set your email and name and may want to set your editor If you prefer you can do the same thing from the command line git config global core editor emacs git config global user email you yourdomain example com git config global user name Your Name Comes Here git config global alias st status git config global alias stat status git config global alias co checkout git config global color diff auto git config global color status true These commands will write to your user s git configuration file gitconfig To set up on another computer you can copy your gitconfig file or run the commands above Other configuration recommended by Yarik In your gitconfig file alias section wdiff diff color words so that git wdiff gives a nicely formatted output of the diff To enforce summaries when doing merges gitconfig file again merge summary true Development workflow You already have your own forked copy of the ipython repository by following Making your own copy fork of ipyt
450. ow everything that needs to be configured when a program starts 6 2 IPython extensions A level above configuration are Python extensions Python modules which modify the behaviour of the shell They are referred to by an importable module name and can be placed anywhere you d normally import from or in TPYTHONDIR extensions 6 2 1 Getting extensions A few important extensions are bundled with IPython Others can be found on the extensions index on the wiki and installed with the install_ext magic function 6 2 2 Using extensions To load an extension while Python is running use the 3lLoad_ext magic In 1 Sload_ext myextension To load it each time Python starts list it in your configuration file c InteractiveShellApp extensions myextension 6 2 3 Writing extensions An Python extension is an importable Python module that has a couple of special functions to load and unload it Here is a template myextension py def load_ipython_extension ipython The ipython argument is the currently active InteractiveShell instance which can be used in any way This allows you to register new magics or aliases for example 6 2 IPython extensions 355 IPython Documentation Release 1 2 1 An Afternoon Hack def unload_ipython_extension ipython If you want your extension to be unloadable put that logic here
451. ow works for the developers We are working hard on continuing to improve it as this was probably IPython s major Achilles heel the lack of proper test coverage made it effectively impossible to do large scale refactoring The full test suite can now be run using the iptest command line program The notion of a task has been completely reworked An Task interface has been created This interface defines the methods that tasks need to implement These methods are now respon sible for things like submitting tasks and processing results There are two basic task types IPython kernel task StringTask this is the old Task object but renamed and the new IPython kernel task MapTask which is based on a function A new interface IPython kernel mapper IMapper has been defined to standardize the idea of a map method This interface has a single map method that has the same syntax as the built in map We have also defined a mapper factory interface that creates objects that implement IPython kernel mapper IMapper for different controllers Both the multiengine and task controller now have mapping capabilties The parallel function capabilities have been reworks The major changes are that i there is now an parallel magic that creates parallel functions ii the syntax for multiple variable follows that of map iii both the multiengine and task controller now have a parallel function implementation All of the parallel computing capabilitie
452. p them visible as you fill in a function s parameters or transfer the information to the pager at the bottom of the screen For the details look at the example notebook O1_notebook_introduction ipynb Other improvements to the Notebook These are some other notable small improvements to the notebook in addition to many bug fixes and minor changes to add polish and robustness throughout The notebook pager the area at the bottom is now resizeable by dragging its divider handle a feature that had been requested many times by just about anyone who had used the notebook system PR 1705 It is now possible to open notebooks directly from the command line for example ipython notebook path will automatically set path as the notebook directory and ipython notebook path foo ipynb will further start with the foo ipynb notebook opened PR 1686 If a notebook directory is specified with notebook dir or with the corresponding configuration flag NotebookManager notebook_dir all kernels start in this directory Fix codemirror clearing of cells with Ctr1 Z PR 1965 Text markdown cells now line wrap correctly in the notebook making them much easier to edit PR 1330 PNG and JPEG figures returned from plots can be interactively resized in the notebook by dragging them from their lower left corner PR 1832 Clear In prompt numbers on Clear All Output For more version control friendly ipynb files we now strip
453. pansion moves to next cell in notebook nbviever alters the order of print and display output print name for Python Notebooks has become uninformative client doesn t use connection file s location in disambiguating interface failing traceback in terminal ipython for first exception Installing in a python 3 3b2 or python 3 3rc1 virtual environment Failure in test Test we re not loading modules on startup that we shouldn t Multiple Notebook Apps cookies not port specific clash with each other running unittest from qtconsole prints output to terminal remote tracebacks broken since 952d0d6 PR 2223 qtconsole does not hightlight tab completion suggestion with custom stylesheet Parsing Tex formula fails in Notebook Parsing Tex formula fails Add argument to run n for custom namespace C m n p don t work in Markdown cells in the notebook time time in ipython notebook producing impossible results schedule tasks on newly arrived engines Allow Notebook HTML JS to send messages to Python code 68 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 2304 1878 2253 1510 2273 2243 2266 2217 2276 2239 2272 2252 2259 2247 2026 2241 2237 2053 1456 2221 2213 1464 2187 2195 2201 2205 1927 2171 2173 2099 2166 ipengine throws KeyError url shell access using will not
454. parent_header b metadata b content b blob zmq identity ies delimiter HMAC signature serialized header dict serialized parent header dict serialized metadata dict serialized content dict extra raw data buffer s Se FE Sh SR SR OR OSE OR The front of the message is the ZeroMQ routing prefix which can be zero or more socket identities This is every piece of the message prior to the delimiter key lt IDS MSG gt In the case of IOPub there should be just one prefix component which is the topic for IOPub subscribers e g pyout display_data Note In most cases the IOPub topics are irrelevant and completely ignored because frontends just sub scribe to all topics The convention used in the Python kernel is to use the msg_type as the topic and possibly extra information about the message e g pyout or stream stdout After the delimiter is the HMAC signature of the message used for authentication If authentication is dis abled this should be an empty string By default the hashing function used for computing these signatures is sha256 Note To disable authentication and signature checking set the key field of a connection file to an empty string The signature is the HMAC hex digest of the concatenation of e A shared key typically the key field of a connection file 386 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hac
455. ped has been updated to the latest version The save magic now has a f flag to force overwriting which makes it much more usable in the notebook where it is not possible to reply to interactive questions from the kernel PR 1937 Use dvipng to format sympy Matrix enabling display of matrices in the Qt console with the sympy printing extension PR 1861 Our messaging protocol now has a reasonable test suite helping ensure that we don t accidentally deviate from the spec and possibly break third party applications that may have been using it We encourage users to contribute more stringent tests to this part of the test suite PR 1627 Use LaTeX to display on output various built in types with the SymPy printing extension PR 1399 2 4 0 13 Series 87 IPython Documentation Release 1 2 1 An Afternoon Hack e Add Gtk3 event loop integration and example PR 1588 e clear_output improvements which allow things like progress bars and other simple animations to work well in the notebook PR 1563 clear_output clears the line even in terminal Python the QtConsole and plain Python as well by printing r to streams clear_output avoids the flicker in the notebook by adding a delay and firing immediately upon the next actual display message display_javascript hides its output_area element so using display to run a bunch of javascript doesn t result in ever growing vertical space e Add simple supp
456. ped the integraton of Python into the Python plugin for Visual Studio 2010 e Google Summer of Code in 2010 we had two students developing prototypes of the new machinery that is now maturing in this release Omar Zapata and Gerardo Guti rrez Development summary moving to Git and Github In April 2010 after one breakage too many with bzr we decided to move our entire development process to Git and Github com This has proven to be one of the best decisions in the project s history as the combination of git and github have made us far far more productive than we could be with our previous tools We first converted our bzr repo to a git one without losing history and a few weeks later ported all open Launchpad bugs to github issues with their comments mostly intact modulo some formatting changes This ensured a smooth transition where no development history or submitted bugs were lost Feel free to use our little Launchpad to Github issues porting script if you need to make a similar transition These simple statistics show how much work has been done on the new release by comparing the current code to the last point it had in common with the 0 10 series A huge diff and 2200 commits make up this cycle git diff git merge base 0 10 2 HEAD wc 1 288019 git log git merge base 0 10 2 HEAD HEAD oneline we 1 2200 Since our move to github 511 issues were closed 226 of which were pull requests and 285
457. perez ipython 2 gt for i in 1 2 3 Cae print i eee 6 7 Outdated configuration information that might still be useful 369 IPython Documentation Release 1 2 1 An Afternoon Hack 370 Chapter 6 Configuration and customization CHAPTER 7 IPython developer s guide This are two categories of developer focused documentation 1 Documentation for developers of Python itself 2 Documentation for developers of third party tools and libraries that use Python This part of our documentation only contains information in the second category Developers interested in working on Python itself should consult our developer information on the Python GitHub wiki 7 1 Working with IPython source code These pages describe a git and github workflow for the Python project There are several different workflows here for different ways of working with Python This is not a comprehensive git reference it s just a workflow for our own project It s tailored to the github hosting service You may well find better or quicker ways of getting stuff done with git but these should get you started For general resources for learning git see gif resources Contents 7 1 1 Install git Overview Debian Ubuntu sudo apt get install git core Fedora sudo yum install git core Windows Download and install msysGit OS X Use the git osx installer 371 IPython Documentation Releas
458. port require In 70 require re def findall pat x bce ctw re is guaranteed to be available Kaaa return re findall pat x you can also pass modules themselves that you already have locally In 71 require time def wait t ke wee time sleep t E EELUEH T Note sync_imports does not allow import foo as bar syntax because the assignment rep resented by the as bar part is not available to the import hook Parallel exceptions In the multiengine interface parallel commands can raise Python exceptions just like serial commands But it is a little subtle because a single parallel command can actually raise multiple exceptions one for each engine the command was run on To express this idea we have a CompositeError exception class that will be raised in most cases The CompositeError class is a special type of exception that wraps one or more other types of exceptions Here is how it works In 78 dview block True In 79 dview execute 1 0 O execute ZeroDivisionError Traceback most recent call last 286 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack gt 1 1 0 ZeroDivisionError integer division or modulo by zero l execute ZeroDivisionError Traceback most recent call last gt 1 1 0 ZeroDivisionError integer division or modulo by zero
459. proved performance 3 uses subcommands for different types of clusters 4 uses argparse for parsing command line options 5 has better support for starting clusters using mpirun 6 has experimental support for starting engines using PBS It can also reuse FURL files by appropriately passing options to its subcommands However this new version of ipcluster should be considered a technology preview We plan on changing the API in significant ways before it is final Full description of the security model added to the docs cd completer show bookmarks if no other completions are available sh profile easy way to give title to prompt assign to variable _prompt_title It looks like this 1 gt _prompt_title sudo sudo 2 gt Bug edit If you do Pedit pasted_block pasted_block variable gets updated with new data so repeated editing makes sense fixes Fix 368719 removed top level debian directory to make the job of Debian packagers easier Fix 291143 by including man pages contributed by Stephan Peijnik from the Debian project Fix 358202 effectively a race condition by properly synchronizing file creation at cluster startup time timeit now handles correctly functions that take a long time to execute even the first time by not repeating them Fix 239054 releasing of references after exiting Fix 341726 thanks to Alexander Clausen Fix 269966 This long standing and very d
460. pts as C_ColorName The list of valid color names is Black Blue Brown Cyan DarkGray Green LightBlue LightCyan LightGray LightGreen LightPurple LightRed NoColor Normal Purple Red White Yellow Finally Python supports the evaluation of arbitrary expressions in your prompt string The prompt strings are evaluated through the syntax of PEP 215 but basically you can use x y to expand the value of x y and for more complicated expressions you can use braces foo x will call function foo and add to it the value of x before putting the result into your prompt For example using prompt_inl com mands getoutput uptime nIn will print the result of the uptime command on each prompt assuming the commands module has been imported in your ipythonrc file Prompt examples The following options in an ipythonrc file will give you IPython s default prompts prompt_inl In prompt_in2 sN Dae prompt_out Out which look like this Tan Ps Ea Out 1 3 In 2 s for a ain 1 273 print i 368 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack These will give you a very colorful prompt with path information prompt_inl C_Red u C_Blue C_Cyan Y1 C_Blue C_LightGreen gt prompt_in2 D gt prompt_out lt gt which look like this fperez ipython 1 gt 1 2 lt 1 gt 3 f
461. pure atomic load balancing is too primitive for your work In these cases you may want to associate some kind of Dependency that describes when where or whether a task can be run In IPython we provide two types of dependencies Functional Dependencies and Graph Dependencies Note It is important to note that the pure ZeroMQ scheduler does not support dependencies and you will see errors or warnings if you try to use dependencies with the pure scheduler Functional Dependencies Functional dependencies are used to determine whether a given engine is capable of running a par ticular task This is implemented via a special Exception class UnmetDependency found in IPython parallel error Its use is very simple if a task fails with an UnmetDependency exception then the scheduler instead of relaying the error up to the client like any other error catches the error and submits the task to a different engine This will repeat indefinitely and a task will never be submitted to a given engine a second time You can manually raise the UnmetDependency yourself but Python has provided some decorators for facilitating this behavior There are two decorators and a class used for functional dependencies In 9 from IPython parallel import depend require dependent require The simplest sort of dependency is requiring that a Python module is available The require decorator lets you define a function that will only run
462. px a numpy random rand 2 2 Parallel execution on engines 0 1 2 3 In 28 px numpy linalg eigvals a Parallel execution on engines 0 1 2 3 Out 0 68 array 0 77120707 0 19448286 Out 1 68 array 1 10815921 0 05110369 Out 2 68 array 0 74625527 0 37475081 Out 3 68 array 0 72931905 0 07159743 5 4 Parallel Magic Commands 289 IPython Documentation Release 1 2 1 An Afternoon Hack In 29 px print hi Parallel execution on engine s all stdout 0 hi stdout 1 hi stdout 2 hi stdout 3 hi Since engines are Python as well you can even run magics remotely In 28 px pylab inline Parallel execution on engine s all stdout 0 Populating the interactive namespace from numpy and matplotlib stdout 1 Populating the interactive namespace from numpy and matplotlib stdout 2 Populating the interactive namespace from numpy and matplotlib stdout 3 Populating the interactive namespace from numpy and matplotlib And once in pylab mode with the inline backend you can make plots and they will be displayed in your frontend if it suports the inline figures e g notebook or qtconsole In 40 px plot rand 100 Parallel execution on engine s all lt plot0 gt lt plot1 gt lt plot2 gt lt plot3 gt Out 0 79 lt matplotlib lines Line2D at 0x10a6286d0 gt O
463. qtconsole non standard handling of a and b Fixes 1561 1574 BUG Ctrl C crashes wx pylab kernel in qtconsole 1573 BUG Ctrl C crashes wx pylab kernel in qtconsole 1590 iPython3 qtconsole doesn t work in Windows 7 602 User test the html notebook 613 Implement Namespace panel section 879 How to handle Javascript output in the notebook 1255 figure show raises an error with the inline backend 1467 Document or bundle a git integrated facility for stripping VCS unfriendly binary data 1237 Kernel status and logout button overlap 1319 Running a cell with ctrl Enter selects text in cell 1571 module member autocomplete should respect __all__ 1566 ipython3 doesn t run in Win7 with Python 3 2 1568 fix PR 1567 1567 Fix openssh_tunnel did not parse port in server 1565 fix AsyncResult abort 1550 Crash when starting notebook in a non ascii path 1552 use os getcwdu in NotebookManager 1554 wrong behavior of the all function on iterators 1541 display_pub flushes stdout err 1539 Asynchrous issue when using clear_display and print x y z 1544 make MultiKernelManager kernel_manager_class configurable 1494 Untrusted Secure Websocket broken on latest chrome dev 1521 only install ipython qtconsole gui script on Windows 1528 Tab completion optionally respects __all___ dir2 cleanup 1527 Making a progress bar work in Python Notebook 1497 __all__ functionality added to dir2 obj
464. r Highest 11 15 2009 11 30 07 PM Auto Auto Core lt Job Name IPEngineSet IV Expand parametric tasks Task Job Details Activity Log Task ID _ Task Name State Command Line sted Resources Start Time PEngine Running lue domainusers bg 1 1 Cores 11 Nblue domainusers bg _ 1 1 Cores WW blue domainusers bg 1 1 Cores jue domainusers bg 1 1 Cores PEngine Running PEngine Running PEngine Running blue domainusers bg 1 1 Cores blue domainusers bg _ 1 1 Cores blue domainusers bg 1 1 Cores PEngine Running PEngine Running PEngine Running 9 PEngine Running 10 PEngine Running blue domainusers bg 1 1 Cores jue domainusers bg 1 1 Cores blue domainusers bg 1 1 Cores blue domainusers bg 1 1 Cores lue domainusers bg _ 1 1 Cores 11 PEngine Running 12 PEngine Running 13 PEngine Running 14 PEngine Running 15 PEngine Running liz Data updated 11 15 2009 11 32 39 PM blue domainusers bg 1 1 Cores blue domainusers bg _ 1 1 Cores bl bl bi bl bl bi bl 8 PEngine Running Ablue domainusersS bg 1 1 Cores bi bl ibl bi bl bl bi 5 10 5 Performing a simple interactive parallel computation Once you have started your Python cluster you can start to use it To do this open up a new Windows Command Prompt and start up Python s interactive shell by typing ipython Then you can create a Di rect View inst
465. r e 2 7 Issues closed in the 0 12 development cycle 139 IPython Documentation Release 1 2 1 An Afternoon Hack 984 985 963 976 929 964 961 953 950 940 948 916 923 920 941 931 921 815 936 937 913 933 930 910 926 928 901 O12 905 909 887 895 Do not expose variables defined at startup to who etc Fixes for parallel code on Python 3 disable calltips in PySide lt 1 0 7 to prevent segfault Getting started on what s new Multiline history Default profile Disable the pager for the test suite Physics extension Add directory for startup files allow setting HistoryManager hist_file with config Monkeypatch Tornado 2 1 1 so it works with Google Chrome 16 Run p https github com ipython ipython pull 901 config magic unordered iteration of AsyncMapResults a couple fixes Follow up to 387dcd6a _rl __doc__ is None with pyreadline read only notebook mode Show invalid config message on TraitErrors during init Fix 481 using custom qt4 input hook Start webbrowser in a thread Prevents lockup with Chrome add dirty trick for readline import on OSX Py3 tests2 Cancel in qt console closeevent should trigger event ignore read only notebook mode Make import checks more explicit in whos reincarnate V cmdline option BUG Set context for font size change shortcuts in ConsoleWidget There is a bug when running the profiler in the magic command prun with pyth
466. rable Exclude frontend shim from docs build use bootstrap disabled instead of ui state disabled Added relative import of RevealExporter to __ init__ py inside exporters module fix HTML capitalization in nbconvert exporter classes fix nbconvert filter validation Get Tracer working after ipapi get replaced with get_ipython use window onbeforeunload for nav away warning don t use parent self in handlers Merge nbconvert into Python restore unsaved changes warning on unload add a dialog when the kernel is auto restarted Add test suite for autoreload extension Catch some pathological cases inside oinspect Display R errors without Python traceback fix magic output add parent to Configurable Remove unexpected keyword parameter to remove_kernel SymPy has changed its recommended way to initialize printing 2 3 Issues closed in the 1 0 development cycle 31 IPython Documentation Release 1 2 1 An Afternoon Hack PR 3486 PR 3483 PR 3482 PR 3381 PR 3450 PR 3477 PR 3466 PR 3465 PR 3463 PR 3456 PR 3453 PR 3393 PR 3440 PR 3431 PR 3425 PR 3433 PR 3298 PR 3428 PR 3420 PR 3416 PR 3413 PR 3364 PR 3390 PR 3403 PR 3401 PR 3373 PR 3307 PR 3392 PR 3369 PR 3370 PR 3372 PR 3378 Add test for non ascii characters in docstrings Inputtransformer Allow classic prompts without space Use an absolute path to iptest because the tests are not always
467. ration system automates this process and allows each command line option to be linked to a particular attribute in the configuration hierarchy that it will override e Configuration files that are themselves valid Python code This accomplishes many things First it becomes possible to put logic in your configuration files that sets attributes based on your operating system network setup Python version etc Second Python has a super simple syntax for accessing hierarchical data structures namely regular attribute access Foo Bar Bam name Third using Python makes it easy for users to import configuration attributes from one configuration file to another 354 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack Fourth even though Python is dynamically typed it does have types that can be checked at runtime Thus a 1 in a config file is the integer 1 while a 1 is a string e A fully automated method for getting the configuration information to the classes that need it at runtime Writing code that walks a configuration hierarchy to extract a particular attribute is painful When you have complex configuration information with hundreds of attributes this makes you want to cry e Type checking and validation that doesn t require the entire configuration hierarchy to be specified statically before runtime Python is a very dynamic language and you don t always kn
468. recent rce db_query started Sgte hourago client_uuid rc session session All jobs started more than an hour ago by clients other than me In 3 recent rce db_query started Sle hourago client_uuid Sne rc session session Result headers for all jobs on engine 3 or 4 In 1 uuids map rc _engines get 3 4 In 2 hist34 rce db_query engine_uuid Sin uuids keys result_header 5 8 4 Cost The advantage of the database backends is of course that large amounts of data can be stored that won t fit in memory The basic DictDB backend is actually to just store all of this information in a Python dictionary This is very fast but will run out of memory quickly if you move a lot of data around or your cluster is to run for a long time Unfortunately the DB backends SQLite and MongoDB right now are rather slow and can still consume large amounts of resources particularly if large tasks or results are being created at a high frequency For this reason we have added NoDB a dummy backend that doesn t actually store any information When you use this database nothing is stored and any request for results will result in a KeyError This obviously prevents later requests for results and task resubmission from functioning but sometimes those nice features are not as useful as keeping Hub memory under control 5 9 Security details of IPytho
469. registered with register_post_execute Warning The API for running code before after the main code block is likely to change soon Both the pre_runcode_hook and the register_post_execute are susceptible to modification as we find a consistent model for both To understand how the code field is executed one must know that Python code can be compiled in one of three modes controlled by the mode argument to the compile builtin single Valid for a single interactive statement though the source can contain multiple lines such as a for loop When compiled in this mode the generated bytecode contains special instructions that trigger the calling of sys displayhook for any expression in the block that returns a value This means that a single statement can actually produce multiple calls to sys displayhook if for example it contains a loop where each iteration computes an unassigned expression would generate 10 calls for i in range 10 L 2 exec An arbitrary amount of source code this is how modules are compiled sys displayhook is never implicitly called 7 2 Messaging in IPython 389 IPython Documentation Release 1 2 1 An Afternoon Hack eval A single expression that returns a value sys displayhook is never implicitly called The code field is split into individual blocks each of which is valid for execution in single mode and then e If there is only a single block it is exec
470. regular issues a full list with links is available for those interested in the details Github s pull requests are a fantastic mech anism for reviewing code and building a shared ownership of the project and we are making enthusiastic use of it Note This undercounts the number of issues closed in this development cycle since we only moved to github for issue tracking in May 2010 but we have no way of collecting statistics on the number of issues closed in the old Launchpad bug tracker prior to that Qt Console IPython now ships with a Qt application that feels very much like a terminal but is in fact a rich GUI that runs an Python client but supports inline figures saving sessions to PDF and HTML multiline editing with syntax highlighting graphical calltips and much more We hope that many projects will embed this widget which we ve kept deliberately very lightweight into their own environments In the future we may also offer a slightly more featureful application with menus and other GUI elements but we remain committed to always shipping this easy to embed widget See the Ot console section of the docs for a detailed description of the console s features and use 2 8 0 11 Series 155 IPython Documentation Release 1 2 1 An Afternoon Hack Python 2 6 5 r265 79063 Apr 16 2010 13 57 41 IPython gt quickref gt Quick reference help object guiref In 1 gt Python s own
471. representations of objects if they are available The Python display system provides a mechanism for specifying PNG or SVG and more representa tions of objects for GUI frontends When you enable matplotlib integration via the smatplotlib magic Python registers convenient PNG and SVG renderers for matplotlib figures so you can embed them in your document by calling display on one or more of them This is especially useful for saving your work 4 5 A Qt Console for Python 235 IPython Documentation Release 1 2 1 An Afternoon Hack In 4 from IPython display import display In 5 plt plot range 5 plots in the matplotlib window In 6 display plt gcf embeds the current figure in the qtconsole In 7 display getfigs embeds all active figures in the qtconsole If you have a reference to a matplotlib figure object you can always display that specific figure In 1 f plt figure In 2 plt plot np rand 100 Out 2 lt matplotlib lines Line2D at 0x7fc6ac03dd90 gt In 3 display f Plot is shown here In 4 plt title A title Out 4 lt matplotlib text Text at 0x7fc6ac023450 gt In 5 display f Updated plot with title is shown here matplotlib inline If you want to have all of your figures embedded in your session instead of calling display you can specify matplotlib inline when you start the console and each time you
472. restrictions on where and when tasks can execute for more complicated load balanced workflows 5 13 4 Data Movement Since the LoadBalancedView does not know where execution will take place explicit data movement methods like push pull and scatter gather do not make sense and are not provided 5 13 Details of Parallel Computing with IPython 337 IPython Documentation Release 1 2 1 An Afternoon Hack 5 13 5 Results AsyncResults Our primary representation of the results of remote execution is the AsyncResult object based on the ob ject of the same name in the built in mult iprocessing pool module Our version provides a superset of that interface The basic principle of the AsyncResult is the encapsulation of one or more results not yet completed Execu tion methods including data movement such as push pull will all return AsyncResults when block False The mp pool AsyncResult interface The basic interface of the AsyncResult is exactly that of the AsyncResult in multiprocessing pool and consists of four methods class AsyncResult The stdlib AsyncResult spec wait timeout Wait until the result is available or until timeout seconds pass This method always returns None ready Return whether the call has completed successful Return whether the call completed without raising an exception Will raise AssertionError if the result is not ready get timeout Return the result when it a
473. rights reserved Redistribution and use in source and binary forms with or without modification are permitted provided that the following conditions are met Redistributions of source code must retain the above copyright notice this list of conditions and the following disclaimer Redistributions in binary form must reproduce the above copyright notice this list of conditions and the following disclaimer in the documentation and or other materials provided with the distribution Neither the name of the IPython Development Team nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission HIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS AS S AND ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO HE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR URPOSE ARE DISCLAIMED IN NO EVENT SHALL THE COPYRIGHT OWNER OR ONTRIBUTORS BE LIABLE FOR ANY DIRECT INDIRECT INCIDENTAL SPECIAL XEMPLARY OR CONSEQUENTIAL DAMAGES INCLUDING BUT NOT LIMITED TO ROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR ROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING R Gl H a UAHA
474. ript Edit code entered from ipython in external editor IPC transport option not taking effect Better error messages in ipengine ipcontroller Cannot send builtin module definitions to IP engines Any reason not to use super Cannot interrupt infinite loops in the notebook clippath_demo py in matplotlib example does not work with inline backend Numbered list in notebook markdown cell renders with Roman numerals instead of numbers Python crashing during startup with AttributeError NoneType object has no attribute 2 nbviewer bug with multi file gists mistyping ed p breaks ed p Glob expansion tests fail on Windows doc issue notebooks that ship with Ipython 13 are written for python 2 x Add Cell gt Run All Keep Going for notebooks Show last modification date of each notebook user request put link to example notebooks in Dashboard grid blanks plots in ipython pylab inline mode interactive Django shell Python gives NameError on dict comprehensions ipython crashes on ctrl c Request nbformat API to load save without changing version Restart kernel message even though kernel is perfectly alive Garbled input text after reverse search on Mac OS X ipdb with separate kernel client pushing stdout to kernel process only 2 3 Issues closed in the 1 0 development cycle 63 IPython Documentation Release 1 2 1 An Afternoon Hack 2180 Have kernel busy overridden only by kernel idle 1188 P
475. ripts Because Twisted and zope interface are now easy installable we can declare them as dependencies in our setupegg py script Python is now compatible with Twisted 2 5 0 and 8 x Added a new example of how to use ipythonl kernel asynclient Initial draft of a process daemon in ipythonl daemon This has not been merged into Python and is still in ipython dev The TaskCont roller now has methods for getting the queue status The TaskResult objects not have information about how long the task took to run We are attaching additional attributes to exceptions _ipython_ that we use to carry additional info around New top level module asyncclient that has asynchronous versions that return deferreds of the client classes This is designed to users who want to run their own Twisted reactor All the clients in client are now based on Twisted This is done by running the Twisted reactor in a separate thread and using the blockingCallFromThread function that is in recent versions of Twisted Functions can now be pushed pulled to from engines using MultiEngineClient push_function andMultiEngineClient pull_function Gather scatter are now implemented in the client to reduce the work load of the controller and improve performance 2 11 0 9 series 191 IPython Documentation Release 1 2 1 An Afternoon Hack e Complete rewrite of the Python docuementation All of the documentation from the Python website
476. rk AT gmail com gt Cocoa GUI frontend architecture Laurent Dufrechou lt laurent dufrechou AT gmail com gt wxPython Python GUI J rgen Stenarson lt jorgen stenarson AT bostream nu gt Maintainer of the PyReadline project which is needed for Python under windows Thomas Kluyver lt takowl AT gmail com gt Port of Python and its necessary ZeroMQ infrastructure to Python3 Python core Evan Patterson lt epatters AT enthought com gt Qt console frontend with ZeroMQ Paul Ivanov lt pi AT berkeley edu gt Python core documentation 419 IPython Documentation Release 1 2 1 An Afternoon Hack e Matthias Bussonnier lt bussonniermatthias AT gmail com gt Python notebook nbviewer nbconvert e Julian Taylor lt jtaylor debian AT googlemail com gt Python core Debian packaging e Brad Froehle lt brad froehle AT gmail com gt Python core 9 1 2 Special thanks The Python project is also very grateful to Bill Bumgarner lt bbum AT friday com gt for providing the DPyGetOpt module that Python used for parsing command line options through version 0 10 Ka Ping Yee lt ping AT lfw org gt for providing the Itpl module for convenient and powerful string interpo lation with a much nicer syntax than formatting through the operator Arnd Baecker lt baecker AT physik tu dresden de gt for his many very useful suggestions and comments and lots of help with testing and documentation checking Many of IPython
477. rk machine with matplotlib plots and everything logging in from home caf s etc gt ipython kernel IPKernelApp To connect another client to this kernel use IPKernelApp existing kernel 12345 json This is actually exactly the same as the subprocess launched by the qtconsole so all the information about connecting to a standalone kernel is identical to that of connecting to the kernel attached to a running console Security Warning Since the ZMQ code currently has no encryption listening on an external facing IP is dan gerous You are giving any computer that can see you on the network the ability to connect to your kernel and view your traffic Read the rest of this section before listening on external ports or running an IPython kernel on a shared machine By default for security reasons the kernel only listens on localhost so you can only connect multiple frontends to the kernel from your local machine You can specify to listen on an external interface by specifying the ip argument gt ipython qtconsole ip 192 168 1 123 240 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack If you specify the ip as 0 0 0 0 or that means all interfaces so any computer that can see yours on the network can connect to the kernel Messages are not encrypted so users with access to the ports your kernel is using will
478. rmatters Ready colorize bug Windows workdir Whitespace cleanup Version info update our version management system to use git Ready for merge Resolved Print shortcut collision with ctrl P emacs binding 172 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack cleanup of exit quit commands for qt console Logo added to sphinx docs Cleanup old code Expose Pygments styles as options HTML Fixes Fix del method exit test Qt frontend shutdown behavior fixes and enhancements Added HTML export Execution refactor Ipy3 preparation 155 152 151 149 Ready startup fix 0 10 1 sge mk_object_info gt object_info Simple bug fix Regular issues 285 630 623 437 583 594 603 597 608 609 611 610 612 621 616 605 302 435 new py in pwd prevents ipython from starting Execute DirectView commands while running LoadBalanced View tasks Users should have autocompletion in the notebook update manpages irunner command line options defer to file extensions Users should see colored text in tracebacks and the pager UnicodeDecodeError ascii codec can t decode byte Oxc2 Organize and layout buttons in the notebook panel sections Implement controls in the Kernel panel section Add kernel status widget back to notebook Implement controls in the Cell section panel Implement Help panel section qtconsole on windows xp can
479. rofile folder The profile folder is a subfolder of your Python directory to find out where it is located run ipython locate To create a new set of default configuration files with lots of information on available options use ipython profile create See also Overview of the Python configuration system in particular Profiles Notebook security Running a public notebook server 4 6 8 Importing py files py files will be imported as a notebook with the same basename but an ipynb extension located in the notebook directory The notebook created will have just one cell which will contain all the code in the py file You can later manually partition this into individual cells using the Edit Split Cell menu option or the Ctrl m keyboard shortcut Note that py scripts obtained from a notebook document using nbconvert_ maintain the structure of the notebook in comments Reimporting such a script back into a notebook will preserve this structure Warning While in simple cases you can roundtrip a notebook to Python edit the Python file and then import it back without loss of main content this is in general not guaranteed to work First there is extra metadata saved in the notebook that may not be saved to the py format And as the notebook format evolves in complexity there will be attributes of the notebook that will not survive a roundtrip through the Python form You should think of
480. rong Renaming a notebook to an existing notebook name overwrites the other file Abort trap in Python Notebook after installing matplotlib issue 3000 2 3 Issues closed in the 1 0 development cycle 61 IPython Documentation Release 1 2 1 An Afternoon Hack 2995 2993 2992 2425 2986 2978 2557 2969 2762 2956 2943 2953 2952 2949 2947 2938 2936 2935 2920 2914 2925 2909 ipython_directive py fails on multiline when prompt number lt 100 File magic file does not work with paths beginning with tilde e g anaconda stuff txt Cell based input for console and qt frontends Liaise with Spyder devs to integrate newer Python requesting help in a loop can damage a notebook v1 0 dev build errors on Arch with Python 3 refactor Insert_cell_at_index ipython command does not work in terminal OSX wxPython osx_cocoa 64bit command gui wx blocks the interpreter Silent importing of submodules differs from standard Python3 2 interpreter s behavior Up arrow key history search gets stuck in QTConsole using nonlocal declaration in global scope causes ipython3 crash qtconsole ignores exec_lines ipython crashes due to atexit From rmagic to an R console docstring pane not showing in notebook Tornado assumes invalid signature for parse_qs on Python 3 1 unable to find python after easy_install pip install Add undo cell deletion menu BUG saving a modifi
481. rrived in OMQ on the client only when it arrived in Python via Client spin so in interactive use this may not be strictly informative Information about the engine engine_id The integer id engine_uuid The UUID of the engine output of the call pyerr Python exception if there was one pyout Python output stderr stderr stream stdout stdout e g print stream And some extended information status either ok or error msg_id The UUID of the message 5 13 Details of Parallel Computing with IPython 339 IPython Documentation Release 1 2 1 An Afternoon Hack after For tasks the time based msg_id dependencies follow For tasks the location based msg_id dependencies While in most cases the Clients that submitted a request will be the ones using the results other Clients can also request results directly from the Hub This is done via the Client s get_result method This method will always return an AsyncResult object If the call was not submitted by the client then it will be a subclass called AsyncHubResult These behave in the same way as an AsyncResult but if the result is not ready waiting on an AsyncHubResult polls the Hub which is much more expensive than the passive polling used in regular AsyncResults The Client keeps track of all results history results metadata 5 13 6 Querying the Hub The Hub sees all traffic that may pass through the schedulers between engines and clients It does this
482. rrives If timeout is not None and the result does not arrive within timeout seconds then TimeoutError is raised If the remote call raised an exception then that exception will be reraised asa RemoteError by get While an AsyncResult is not done you can check on it with its ready method which will return whether the AR is done You can also wait on an AsyncResult with its wait method This method blocks until the result arrives If you don t want to wait forever you can pass a timeout in seconds as an argument to wait wait will always return None and should never raise an error ready and wait are insensitive to the success or failure of the call After a result is done successful will tell you whether the call completed without raising an exception If you actually want the result of the call you can use get Initially get behaves just like wait in that it will block until the result is ready or until a timeout is met However unlike wait get will raise a TimeoutError if the timeout is reached and the result is still not ready If the result arrives before the timeout is reached then get will return the result itself if no exception was raised and will raise an exception if there was Here is where we start to expand on the multiprocessing interface Rather than raising the original exception a RemoteError will be raised encapsulating the remote exception with some metadata If the AsyncR
483. rsion0 9 txt Fix ipython gt ipython typo add missing KernelManager to ConsoleApp class list Consolidate host IP detection in utils localinterfaces Correct docstring of ipython py avoid string version comparisons in external qt this should address the failure in 2732 utils data Use list comprehension for uniq_stable add jinja to install docs setup py Miscellaneous docs fixes 2 3 Issues closed in the 1 0 development cycle 39 IPython Documentation Release 1 2 1 An Afternoon Hack PR 2811 PR 2820 PR 2818 PR 2828 PR 2829 PR 2830 PR 2718 PR 2816 PR 2817 PR 2537 PR 2815 PR 2808 PR 2812 PR 2810 PR 1671 PR 2804 PR 2789 PR 2805 PR 2775 PR 2340 PR 2799 PR 2800 PR 2801 PR 2795 PR 2796 PR 2724 PR 2687 PR 2788 PR 2649 PR 2753 PR 2783 PR 2333 Still more KernelManager cleanup add to greedy completer delims log user tracebacks in the kernel INFO level Clean up notebook Javascript avoid comparison error in dictdb hub history BUG Opening parenthesis after non callable raises ValueError try to fallback to pysqlite2 dbapi2 as sqlite3 in core history in edit don t save last_call unless last call succeeded change ol format order Organize example notebooks update release authors improve patience for slow Hub in client tests remove nonfunctional la short arg in cython magic remove dead utils upgradedir
484. rt set_parent 2925 get monospace pager back fix celltoolbar layout on FF 38 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 2904 PR 2908 PR 2905 PR 2900 PR 2893 PR 2884 PR 2882 PR 2883 PR 2855 PR 2874 PR 2819 PR 2868 PR 2877 PR 2880 PR 2878 PR 2599 PR 2792 PR 2872 PR 2869 PR 2708 PR 2854 PR 2864 PR 2852 PR 2863 PR 2861 PR 2850 PR 2859 PR 2831 PR 2844 PR 2849 PR 2839 PR 2841 Skip remaining IPC test on Windows fix last remaining KernelApp reference fix a few remaining KernelApp IPKernelApp changes Don t assume test case for time will finish in 0 time exclude fabfile from tests Correct import for kernelmanager on Windows Utils cleanup Don t call ast fix_missing_locations unless the AST could have been modified time it magic Implement minutes hour formatting and time cell magic Empty cell warnings tweak history prefix search up p in qtconsole Import performance minor css fixes update examples docs with kernel move Pass host environment on to kernel func_kw_complete for builtin and cython with embededsignature True using docstring Add key unique to history_request protocol fix payload keys Fixing styling of toolbar selects on FF Less css Move kernel code into Python kernel Fix Yrun t N lt N gt TypeError future pyzmq compatibility whatsnew Vve
485. rting it and submitting a pull request on our development site Currently the quarantine directory contains clearcmd py envpersist py ext_rescapture py InterpreterExec py ipy_app_completers py ipy_autoreload py ipy_completers py ipy_editors py ipy_exportdb py ipy_extutil py ipy_fsops py ipy_gnuglobal py ipy_greedycompleter py ipy_jot py ipy_lookfor py ipy_profile_doctest py ipy_pydb py ipy_rehashdir py ipy_render py ipy_server py ipy_signals py ipy_synchronize_with py ipy_system_conf py ipy_which py ipy_winpdb py ipy_workdir py jobctrl py ledit py pspersistence py win32clip py 162 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Deathrow These packages may be harder to update or make most sense as third party libraries Some of them are completely obsolete and have been already replaced by better functionality we simply haven t had the time to carefully weed them out so they are kept here for now Others simply require fixes to code that the current core team may not be familiar with If a tool you were used to is included here we encourage you to contact the dev list and we can discuss whether it makes sense to keep it in Python if it can be maintained Currently the deathrow directory contains astyle py ipy_defaults py ipy_vimserver py dtutils py ipy_kitcfg py numeric_formats py Gnuplot2 py ipy_legacy py numutils py Gnupl
486. rts and patches to work around the exception handling idiosyncracies of WxPython Readline and color support for Windows Jeffrey Collins lt Jeff Collins AT vexcel com gt Bug reports Much improved readline support includ ing fixes for Python 2 3 Dryice Liu lt dryice AT liu com cn gt FreeBSD port 9 1 Credits 421 IPython Documentation Release 1 2 1 An Afternoon Hack Mike Heeter lt korora AT SDF LONESTAR ORG gt Christopher Hart lt hart AT caltech edu gt PDB integration Milan Zamazal lt pdm AT zamazal org gt Emacs info Philip Hisley lt compsys AT starpower net gt Holger Krekel lt pyth AT devel trillke net gt Tab completion lots more Robin Siebler lt robinsiebler AT starband net gt Ralf Ahlbrink lt ralf_ahlbrink AT web de gt Thorsten Kampe lt thorsten AT thorstenkampe de gt Fredrik Kant lt fredrik kant AT front com gt Windows setup Syver Enstad lt syver en AT online no gt Windows setup Richard lt rxe AT renre europe com gt Global embedding Hayden Callow lt h callow AT elec canterbury ac nz gt Gnuplot py 1 6 compatibility Leonardo Santagada lt retype AT terra com br gt Fixes for Windows installation Christopher Armstrong lt radix AT twistedmatrix com gt Bugfixes Francois Pinard lt pinard AT iro umontreal ca gt Code and documentation fixes Cory Dodt lt cdodt AT fcoe k12 ca us gt Bug reports and Windows ideas Patches for Windows in staller Olivier Aubert lt oaubert
487. run from IPYTHONDIR enable 2x retina display Flatten Python frontend pass config to subapps Kernel fails to start when username has non ascii characters Add HTCondor bindings to Python parallel fix typo closes 3462 Notice for users who disable javascript fix cell execution in firefox closes 3447 WIP bootstrapify Fix installing mathjax from downloaded file via command line Provide means for starting the Qt console maximized and with the menu bar hidden base IPClusterApp inherits from BaseIPythonApp Update Pythonexternalpath__init__ py Some fixes in Python Sphinx directive process escapes in mathjax thansk gt thanks Fix doc principle not principal more unique filename for test Inject requirejs in notebook and start using it Fix paste with blank lines fix creating config objects from dicts rollback 3358 make cookie_secret configurable switch default ws_url logic to js side Restore anchor link on h2 h6 Use different treshold for auto scroll in output normalize unicode notebook filenames base default cookie name on request host port disable CodeMirror drag drop on Safari 32 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 3358 PR 3371 PR 3366 PR 3382 PR 3330 PR 3319 PR 3384 PR 3387 PR 3357 PR 3351 PR 3377 PR 3340 PR 3328 PR 3368 PR 3356 PR 3336 PR 3325 PR 3342 PR 3354 PR 334
488. ry at ipython github Note that we ve used git for the URL rather than git The git URL is read only This means we that we can t accidentally or deliberately write to the upstream repo and we are only going to use it to merge into our own code Just for your own satisfaction show yourself that you now have a new remote with git remote v show giving you something like upstream git github com ipython ipython git fetch upstream git github com ipython ipython git push origin git github com your user name ipython git fetch origin git github com your user name ipython git push Configure git Overview git config global user email you yourdomain example com git config global user name Your Name Comes Here In detail This is to tell git who you are for labeling any changes you make to the code The simplest way to do this is from the command line git config global user email you yourdomain example com git config global user name Your Name Comes Here This will write the settings into your git configuration file a file called gitconfig in your home direc tory Advanced git configuration You might well benefit from some aliases to common commands 376 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack For example you might well want to be able to shorten git check
489. s blogs etc This excludes HTML headers e to latex Latex export This generates NOTEBOOK_NAME tex file ready for export You can automatically run latex on it to generate a PDF by adding post PDF template article default Latex article derived from Sphinx s howto template template book Latex book derived from Sphinx s manual template template basic Very basic latex output mainly meant as a starting point for custom templates 252 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack e to slides This generates a Reveal js HTML slideshow It must be served by an HTTP server The easiest way to get this is to add post serve on the command line e to markdown Simple markdown output Markdown cells are unaffected and code cells are placed in triple backtick blocks amp to rst Basic reStructuredText output Useful as a starting point for embedding notebooks in Sphinx docs e to python Convert a notebook to an executable Python script This is the simplest way to get a Python script out of a notebook If there were any magics in the notebook this may only be executable from an Python session Note nbconvert uses pandoc to convert between various markup languages so pandoc is a dependency of most nbconvert transforms excluding Markdown and Python The output file created by n
490. s 1 ppn 4 PBS q queue cd SPBS_O_WORKDIR export PATH SHOME usr local bin export PYTHONPATH SHOME usr local lib python2 7 site packages ipcontroller profile dir profile_dir Once you have created these scripts save them with names like pbs engine template Now you can load them into the ipcluster_config with 5 2 Starting the Python controller and engines 269 IPython Documentation Release 1 2 1 An Afternoon Hack c PBSEngineSetLauncher batch_template_file pbs engine template c PBSControllerLauncher batch_template_file pbs controller template Alternately you can just define the templates as strings inside ipcluster_config Whether you are using your own templates or our defaults the extra configurables available are the number of engines to launch n and the batch system queue to which the jobs are to be submitted queue These are configurables and can be specified in ipcluster_config c PBSLauncher queue veryshort q c IPClusterEngines n 64 Note that assuming you are running PBS on a multi node cluster the Controller s default behavior of lis tening only on localhost is likely too restrictive In this case also assuming the nodes are safely behind a firewall you can simply instruct the Controller to listen for connections on all its interfaces by adding in ipcontroller_config c HubFactory ip 1 You can now ru
491. s also useful on systems with shared filesystems to run the engines in some scratch directory This can be set with c IPEngineApp work_dir u path to scratch 5 3 IPython s Direct interface The direct or multiengine interface represents one possible way of working with a set of Python engines The basic idea behind the multiengine interface is that the capabilities of each engine are directly and ex plicitly exposed to the user Thus in the multiengine interface each engine is given an id that is used to identify the engine and give it work to do This interface is very intuitive and is designed with interactive usage in mind and is the best place for new users of Python to begin 5 3 1 Starting the IPython controller and engines To follow along with this tutorial you will need to start the Python controller and four Python engines The simplest way of doing this is to use the ipcluster command ipcluster start n 4 For more detailed information about starting the controller and engines see our introduction to using Python for parallel computing 5 3 2 Creating a DirectView instance The first step is to import the Python TPython parallel module and then create a Client instance In 1 from IPython parallel import Client In 2 re Client This form assumes that the default connection information stored in ipcontroller client json found in IPYTHONDIR profile_default security
492. s are separated by single space for convenient passing to system commands The n property return one string where the lines are separated by a newline i e the original output of the function If the items in string list are file names p can be used to get a list of path objects for convenient file manipulation 4 4 IPython as a system shell 233 IPython Documentation Release 1 2 1 An Afternoon Hack 4 5 A Qt Console for IPython We now have a version of Python using the new two process ZeroMQ Kernel running in a PyQt GUI This is a very lightweight widget that largely feels like a terminal but provides a number of enhancements only possible in a GUI such as inline figures proper multiline editing with syntax highlighting graphical calltips and much more interactive _images qtconsole png Fig 4 1 The Qt console for Python using inline matplotlib plots To get acquainted with the Qt console type guiref to see a quick introduction of its main features The Qt frontend has hand coded emacs style bindings for text navigation This is not yet configurable Tip Since the Qt console tries hard to behave like a terminal by default it immediately executes single lines of input that are complete If you want to force multiline input hit Ctrl Enter at the end of the first line instead of Enter and it will open a new line for input At any point in a multiline block you can force its execu
493. s closed in the 0 11 development cycle 179 IPython Documentation Release 1 2 1 An Afternoon Hack 111 Iterator version of TaskClient map that returns results as they become available 109 WinHPCLauncher is a hard dependency that causes errors in the test suite 86 Make IPython work with multiprocessing 15 Implement SGE support in ipcluster 3 Implement PBS support in ipcluster 53 Internal Python error in the inspect module 74 Manager from multiprocessing module hangs ipythonx but not ipython 51 Out not working with ipythonx 201 use session send throughout zmq code 115 multiline specials not defined in 0 11 branch 93 when looping cursor appears at leftmost point in newline 133 whitespace after Source introspection 50 Ctrl C with gthread on Windows causes uncaught IOError 65 Do not use message attributes in exceptions deprecated in 2 6 76 syntax error when raise is inside except process 107 bdist_rpm causes traceback looking for a non existant file 113 initial magic question mark fails before wildcard 128 Pdb instance has no attribute curframe 139 running with pylab pollutes namespace 140 malloc error during tab completion of numpy array member functions starting with c 153 ipy_vimserver traceback on Windows 154 using ipython in Slicer3 show how os environ HOME is not defined 185 show blocks in pylab mode with ipython 0 10 1 189 Crash on tab completion 274 bashism in
494. s from ipython dev have been merged into Python proper This resulted in the following new subpackages IPython kernel IPython kernel core IPython config IPython tools and IPython testing As part of merging in the ipythonI dev stuff the setup py script and friends have been completely refactored Now we are checking for dependencies using the approach that matplotlib uses The documentation has been completely reorganized to accept the documentation from ipython dev We have switched to using Foolscap for all of our network protocols in I Python kernel This gives us secure connections that are both encrypted and authenticated 2 11 0 9 series 189 IPython Documentation Release 1 2 1 An Afternoon Hack We have a brand new COPYING txt files that describes the Python license and copyright The biggest change is that we are putting The Python Development Team as the copyright holder We give more details about exactly what this means in this file All developer should read this and use the new banner in all Python source code files sh profile foo runs foo as system command no need to do foo anymore String lists now support sort field nums True method to easily sort system command output Try it witha ls l a sort 1 nums 1 cpaste foo now assigns the pasted block as string list instead of string The ipcluster script now run by default with no security This is done because the main usage
495. s history Do cd lt TAB gt to conveniently view the directory history Automatic parentheses and quotes These features were adapted from Nathan Gray s LazyPython They are meant to allow less typing for common situations Automatic parentheses Callable objects i e functions methods etc can be invoked like this notice the commas between the arguments In 1 callable_ob argl arg2 arg3 a aa gt callable_ob argl arg2 arg3 You can force automatic parentheses by using as the first character of a line For example In 2 globals becomes globals Note that the lt MUST be the first character on the line This won t work In 3 print globals syntax error In most cases the automatic algorithm should work so you should rarely need to explicitly invoke One notable exception is if you are trying to call a function with a list of tuples as arguments the parenthesis will confuse IPython In 4 zip 1 2 3 4 5 6 won t work but this will work In 5 zip 1 2 3 4 5 6 i gt zip 1 2 3 4 5 6 Out 5 1 4 2 5 3 6 Python tells you that it has altered your command line by displaying the new command line preceded by gt e g In 6 callable list Ta gt callable list Automatic quoting You can force automatic quoting of a function s arguments by using
496. s into the API Now these methods simple take a dict or sequence This has also allowed us to get rid of the xA11 methods like pushA11 and pul1A11 These things are now handled with the targets keyword argument that defaults to all The MultiEngineClient magicTargets has been renamed to MultiEngineClient targets All methods in the MultiEngine interface now accept the optional keyword argument block Renamed RemoteController to MultiEngineClient and TaskController to TaskClient Renamed the top level module from api to client Most methods in the multiengine interface now raise a CompositeError exception that wraps the user s exceptions rather than just raising the raw user s exception Changed the setupNS and resultNames in the Task class to push and pull 192 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 2 12 0 8 series 2 12 1 Release 0 8 4 This was a quick release to fix an unfortunate bug that slipped into the 0 8 3 release The twisted option was disabled as it turned out to be broken across several platforms 2 12 2 Release 0 8 3 e pydb is now disabled by default due to run d problems You can enable it by passing pydb command line argument to Python Note that setting it in config file won t work 2 12 3 Release 0 8 2 e pushd popd behave differently now pushd foo pushes CURRENT directory and jumps to
497. s newer features are a result of discussions with him Obviously Guido van Rossum and the whole Python development team for creating a great language for interactive computing Fernando would also like to thank Stephen Figgins lt fig AT monitor net gt an O Reilly Python editor His October 11 2001 article about IPP and LazyPython was what got this project started You can read it at http www onlamp com pub a python 2001 10 1 1 pythonnews html 9 1 3 Sponsors We would like to thank the following entities which at one point or another have provided resources and support to Python Enthought http www enthought com for hosting Python s website and supporting the project in various ways over the years including significant funding and resources in 2010 for the development of our modern ZeroMQ based architecture and Qt console frontend Google for supporting Python through Summer of Code sponsorships in 2005 and 2010 Microsoft Corporation for funding in 2009 the development of documentation and examples of the Windows HPC Server 2008 support in Python s parallel computing tools The Nipy project http nipy org for funding in 2009 a significant refactoring of the entire project codebase that was key Ohio Supercomputer Center part of Ohio State University Research Foundation and the Department of Defense High Performance Computing Modernization Program HPCMP for sponsoring work in 2009 on the ipcluster scrip
498. sage now have execution_count 32 dreload produces spurious traceback when numpy is involved 1457 1613 1609 1531 1607 1608 1421 1599 1201 1602 1603 1604 1601 1600 Update deepreload to use a rewritten knee py Fixes dreload numpy allow map parallel function for single engine views exit notebook cleanly on SIGINT SIGTERM Function keyword completion fails if cursor is in the middle of the complete parentheses cleanup sqlitedb temporary db file after tests don t rely on timedelta total_seconds in AsyncResult ipython32 run d breaks with NameError name execfile is not defined Fix for run d on Python 3 env magic fails with Python 3 2 Fix env magic on Python 3 Remove python3 profile Exclude IPython quarantine from installation Security file is not removed after shutdown by Ctrl C or kill INT Specify encoding for io open in notebook_reformat tests 116 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 1605 1452 Small fixes for Animation and Progress notebook Bug fix for approval 13 Improve robustness and debuggability of test suite 70 Python should prioritize __all___ during tab completion 1529 1475 1564 1548 1535 1522 1399 1597 1490 1487 1545 1480 1588 1595 1526 1594 1556 1499 1502 1445 1484 1538 1546 1524 1559 1563 1558
499. se examples we summarize the key features of IPython s parallel architecture that have been Second digit demonstrated 4200 3600 3000 2400 First digit 1800 1200 600 5 11 2 Conclusion e Serial code can be parallelized often with only a few extra lines of code We have used the DirectView and LoadBalancedView classes for this purpose e The resulting parallel code can be run without ever leaving the Python s interactive shell e Any data computed in parallel can be explored interactively through visualization or further numerical calculations 326 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack e We have run these examples on a cluster running RHEL 5 and Sun GridEngine Python s built in support for SGE and other batch systems makes it easy to get started with Python s parallel capabilities 5 12 DAG Dependencies Often parallel workflow is described in terms of a Directed Acyclic Graph or DAG A popular library for working with Graphs is NetworkX Here we will walk through a demo mapping a nx DAG to task dependencies The full script that runs this demo can be found in examples parallel dagdeps py 5 12 1 Why are DAGs good for task dependencies The G in DAG is Graph A Graph is a collection of nodes and edges that connect the nodes For our purposes each node would be a task and each edge would be a dependency Th
500. security dir of a profile then you need not specify its path directly only the profile assumes the path exists otherwise you must create it first engine host n engine host n scp controller host ipython profile_default security ipcontroller e ipengine profile ssh Of course if you fetch the file into the default profile no arguments must be passed to ipengine at all e Note that ipengine did not specify the ip argument In general it is unlikely for any connection information to be specified at the command line to ipengine as all of this information should be contained in the connection file written by ipcontroller 274 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack Make JSON files persistent At fist glance it may seem that that managing the JSON files is a bit annoying Going back to the house and key analogy copying the JSON around each time you start the controller is like having to make a new key every time you want to unlock the door and enter your house As with your house you want to be able to create the key or JSON file once and then simply use it at any point in the future To do this the only thing you have to do is specify the reuse flag so that the connection information in the JSON files remains accurate ipcontroller reuse Then just copy the JSON files over the first time and you are set You can st
501. sed in the notebook have been updated significantly e updates to jQuery 2 0 and jQueryUI 1 10 e Update CodeMirror to 3 14 e Twitter Bootstrap 2 3 for layout e Font Awesome 3 1 for icons e highlight js 7 3 for syntax highlighting e marked 0 2 8 for markdown rendering e require js 2 1 for loading javascript Some relevant changes that are results of this e Markdown cells now support GitHub flavored Markdown GFM which includes python code blocks and tables e Notebook UI behaves better on more screen sizes e Various code cell input issues have been fixed Kernel The kernel code has been substantially reorganized New features in the kernel e Kernels support ZeroMQ IPC transport not just TCP 2 2 1 0 Series 11 IPython Documentation Release 1 2 1 An Afternoon Hack e The message protocol has added a top level metadata field used for information about messages e Add a data_pub message that functions much like display_pub but publishes raw usually pickled data rather than representations e Ensure that sys stdout encoding is defined in Kernels e Stdout from forked subprocesses should be forwarded to frontends instead of crashing IPEP 13 The KernelManager has been split into a KernelManager anda KernelClient The Manager owns a kernel and starts signals restarts it There is always zero or one KernelManager per Kernel Clients communicate with Kernels via zmq channels and there c
502. shes if a profile is used 56 Have cpaste accept also Ctrl D as a termination marker 849 806 691 688 866 673 863 800 858 840 843 836 839 Command line options to not override profile options Provide single port connection to kernels wishlist Automatically find existing kernel local security vulnerability all ports visible to any local user DistributionNotFound on running ipython 0 11 on Windows XP x86 raw_input appears to be round robin for qtconsole Graceful degradation when home directory not writable Timing scripts with run t N lt N gt fails on report output Typing continue makes ipython0 11 crash all processes run on one CPU core import braces crashes ipython Strange Output after Python Install Qtconsole segfaults when mouse exits window with active tooltip 150 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 827 Add support for checking several limits before running task on engine 826 Add support for creation of parallel task when no engine is running 832 Improve error message for logstop 831 Ylogstart in read only directory forbid any further command 814 ipython does not start DistributionNotFound 794 Allow gt 1 controller per profile 820 Tab Completion feature 812 Qt console crashes on Ubuntu 11 10 816 Import error using Python 2 7 and dateutil2 0 No module named _thread 756 qtconsole Windows fails to pri
503. so that it can track state allowing multiple clients to retrieve results of computations submitted by their peers as well as persisting the state to a database queue_status You can check the status of the queues of the engines with this command result_status check on results purge_results forget results conserve resources 5 13 7 Controlling the Engines There are a few actions you can do with Engines that do not involve execution These messages are sent via the Control socket and bypass any long queues of waiting execution jobs abort Sometimes you may want to prevent a job you have submitted from actually running The method for this is abort It takes a container of msg_ids and instructs the Engines to not run the jobs if they arrive The jobs will then fail with an AbortedTask error clear You may want to purge the Engine s namespace of any data you have left in it After running clear there will be no names in the Engine s namespace shutdown You can also instruct engines and the Controller to terminate from a Client This can be useful when a job is finished since you can shutdown all the processes with a single command 340 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack 5 13 8 Synchronization Since the Client is a synchronous object events do not automatically trigger in your interactive session you must poll the OMQ sockets for incomin
504. splay an PNG JPEG image given raw data When this object is returned by an expression or passed to the display function it will result in the image being displayed in the frontend An image In 2 data unicode str or bytes The raw data or a URL to download the data from Out 2 url unicode A DT ba Aner Vand bha dnibna funm SVG images are also supported out of the box since modern browsers do a good job of rendering them In 3 from IPython core display import SVG SVG filename python logo svg python Out 3 Fig 2 1 The new notebook tooltips 76 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Cell magics We have completely refactored the magic system finally moving the magic objects to standalone indepen dent objects instead of being the mixin class we d had since the beginning of Python PR 1732 Now a separate base class is provided in TPython core magic Magics that users can subclass to create their own magics Decorators are also provided to create magics from simple functions without the need for object orientation Please see the Magic command system docs for further details All builtin magics now exist in a few subclasses that group together related functionality and the new IPython core magics package has been created to organize this into smaller files This cleanup was the last major piece of deep refactoring needed from the
505. ss to your data and functions and you can resume program execution once you are done with the interactive part perhaps to stop again later as many times as needed The following code snippet is the bare minimum you need to include in your Python programs for this to work detailed examples follow later from IPython import embed embed this call anywhere in your program will start IPython Note As of 0 13 you can embed an Python kernel for use with qtconsole etc via IPython embed_kernel instead of Python embed It should function just the same as reg ular embed but you connect an external frontend rather than Python starting up in the local terminal You can run embedded instances even in code which is itself being run at the Python interactive prompt with run lt filename gt Since it s easy to get lost as to where you are in your top level Python or in your embedded one it s a good idea in such cases to set the in out prompts to something different for the embedded instances The code examples below illustrate this You can also have multiple Python instances in your program and open them separately for example with different options for data presentation If you close and open the same instance multiple times its prompt counters simply continue from each execution to the next Please look at the docstrings in the embed module for more details on the use of this system The followin
506. st basic such command is via Ctrl which restarts the kernel This can be done in the middle of a blocking execution The frontend can also know via a heartbeat mechanism that the kernel has died This means that the frontend can safely restart the kernel Multiple Consoles Since the Kernel listens on the network multiple frontends can connect to it These do not have to all be qt frontends any Python frontend can connect and run code When you start ipython qtconsole there will be an output line like IPKernelApp To connect another client to this kernel use IPKernelApp existing kernel 12345 json Other frontends can connect to your kernel and share in the execution This is great for collaboration The existing flag means connect to a kernel that already exists Starting other consoles with that flag will not try to start their own kernel but rather connect to yours kerne1 12345 json is a small JSON file with the ip port and authentication information necessary to connect to your kernel By default this file will be in your default profile s security directory If it is somewhere else the output line will print the full path of the connection file rather than just its filename If you need to find the connection info to send and don t know where your connection file lives there are a couple of ways to get it If you are already running an Python console connected to the kernel you can use t
507. started System shell access Any input line beginning with a character is passed verbatim minus the of course to the underlying operating system For example typing 1s will run ls in the current directory Manual capture of command output You can assign the result of a system command to a Python variable with the syntax myfiles 1s This gets machine readable output from stdout e g without colours and splits on newlines To explicitly get this sort of output without assigning to a variable use two exclamation marks 1s or the sx magic command The captured list has some convenience features myfiles normyfiles s returns a string delimited by newlines or spaces respectively my files p produces path objects from the list items See String lists for details Python also allows you to expand the value of python variables when making system calls Wrap variables or expressions in braces In 1 pyvar Hello world In 2 echo A python variable pyvar A python variable Hello world In 3 import math In 4 x 8 In 5 echo math factorial x 40320 For simple cases you can alternatively prepend to a variable name In 6 echo Ssys argv home fperez usr bin ipython In 7 echo A system variable SSHOME Use for literal A system variable home fperez 216 Chapter 4 Using IPython for interactive work IPython Documentation Rel
508. stderr variables named _ or __ exhibit incorrect behavior add an x or similar to htmlnotebook pager Several 404 errors for js files Firefox syntax highlighting in chrome on OSX 10 8 3 Latest dev version hangs on page load 58 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 3283 3279 2727 3268 3273 3272 3260 3247 3243 3265 3257 3253 3248 3012 2390 3236 3214 2980 3128 3198 3218 3177 2563 3207 3178 3204 1789 3202 3199 2688 3192 3179 ipython dev retains directory information after directory change custom css is not overridden in the dev IPython 1 0 run m doesn t support relative imports GFM triple backquote and unknown language Suppressing all plot related outputs Backspace while completing load previous page Js error in savewidget scrollbar in notebook when not needed notebook option to view json source from browser 404 errors when running Python 1 0dev setup py not finding submodules Incorrect Qt and PySide version comparison Cell magics broken in Qt console Problems with the less based style min css Image width height don t work in embedded images cannot set TerminallPythonApp log_format notebook kernel dies if started with invalid parameter Remove HTMLCell qtconsole hangs on importing pylab using X forwarding Hitting recursive depth causing all notebook pa
509. submitted via MUX queue whose results have not been received tasks tasks that are known to have been submitted to the engine but have not completed Note that with the pure zmq scheduler this wil always be O Message type queue_reply content status Yok or error if verbose False O completed 1 queuve 7 tasks 0 if verbose True t I completed abcd 1234 queuve 58008 tasks Clients can request individual results directly from the hub This is primarily for gathering results of execu tions not submitted by the requesting client as the client will have all its own results already Requests are made by msg_id and can contain one or more msg_id An additional boolean key statusonly can be used to not request the results but simply poll the status of the jobs Message type result_request content msg ids wad sae ly amp Last Of strs targets 1 2 3 list of int ids or uuids statusonly False bool The result_request reply contains the content objects of the actual execution reply messages If statusonly True then there will be only the pending and completed lists Message type result_reply 404 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack content status ok else error if ok acbd
510. t there are for now no additional data fields This happens when the kernel was interrupted by a signal 7 2 Messaging in IPython 391 IPython Documentation Release 1 2 1 An Afternoon Hack Object information One of IPython s most used capabilities is the introspection of Python objects in the user s namespace typically invoked via the and characters which in reality are shorthands for the 3pinfo magic This is used often enough that it warrants an explicit message type especially because frontends may want to get object information in response to user keystrokes like Tab or F1 besides from the user explicitly typing code like x Message type ob ject_info_request content The possibly dotted name of the object to be searched in all relevant namespaces oname str The level of detail desired The default 0 is equivalent to typing x at the prompt 1 is equivalent to x detail_level int The returned information will be a dictionary with keys very similar to the field names that Python prints at the terminal Message type ob ject_info_reply content The name the object was requested under name str Boolean flag indicating whether the named object was found or not If it s false all other fields will be empty Found bool Flags for magics and system aliases rsmagic bool isalias z bool The
511. t See the official Python documentation for more on the basics of this interface Our AsyncResult objects add a number of convenient features for working with parallel results beyond what is provided by the original AsyncResult get_dict First is AsyncResult get_dict which pulls results as a dictionary keyed by engine_id rather than a flat list This is useful for quickly coordinating or distributing information about all of the engines As an example here is a quick call that gives every engine a dict showing the PID of every other engine In 10 ar re apply_async os getpid In 11 pids ar get_dict In 12 rce pid_map pids This trick is particularly useful when setting up inter engine communication as in IPython s examples parallel interengine examples 5 6 2 Metadata IPython parallel tracks some metadata about the tasks which is stored in the Client metadata dict The AsyncResult object gives you an interface for this information as well including timestamps stdout err and engine IDs Timing Python tracks various timestamps as datetime objects and the AsyncResult object has a few properties that turn these into useful times in seconds as floats For use while the tasks are still pending e ar elapsed is just the elapsed seconds since submission for use before the AsyncResult is com plete e ar progress is the number of tasks that have completed Fractional progress would
512. t in the auto generated config files suggestion dependency checking for pyqt for Windows installer broken documentation link on download page Test failures in Python parallel 152 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 2 8 0 11 Series 2 8 1 Release 0 11 Python 0 11 is a major overhaul of Python two years in the making Most of the code base has been rewritten or at least reorganized breaking backward compatibility with several APIs in previous versions It is the first major release in two years and probably the most significant change to Python since its inception We plan to have a relatively quick succession of releases as people discover new bugs and regressions Once we iron out any significant bugs in this process and settle down the new APIs this series will become Python 1 0 We encourage feedback now on the core APIs which we hope to maintain stable during the 1 0 series Since the internal APIs have changed so much projects using Python as a library as opposed to end users of the application are the most likely to encounter regressions or changes that break their existing use patterns We will make every effort to provide updated versions of the APIs to facilitate the transition and we encourage you to contact us on the development mailing list with questions and feedback Chris Fonnesbeck recently wrote an excellent post that highlights some of our major
513. t least three Schedulers These devices are primarily for relaying messages between clients and engines but the Hub needs to see those messages for its own purposes Since no Python code may exist between the two sockets in a queue all messages sent through these queues both directions are also sent via a PUI B socket to a monitor which allows the Hub to monitor queue traffic without interfering 412 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack with it For tasks the engine need not be specified Messages sent to the ROUTER socket from the client side are assigned to an engine via ZMQ s DEALER round robin load balancing Engine replies are directed to specific clients via the IDENTITY of the client which is received as a prefix at the Engine For Multiplexing ROUTER is used for both in and output sockets in the device Clients must specify the destination by the zmq IDENTITY of the ROUTER socket connected to the downstream end of the device At the Kernel level both of these ROUTER sockets are treated in the same way as the REP socket in the serial version except using ZMQStreams instead of explicit sockets Execution can be done in a load balanced engine agnostic or multiplexed engine specified manner The sockets on the Client and Engine are the same for these two actions but the scheduler used determines the actual behavior This routing is done via the
514. t required with automagic on It also maintains a list of visited directories use Sdhist to see it and allows direct switching to any of them Type cd for more details Spushd Spopd and Sdirs are provided for directory stack handling 4 4 4 Environment variables env can be used to remember environment variable manipulations Examples Senv Show all environment variables senv VISUAL jed set VISUAL to jed env PATH foo append foo to PATH Senv PATH bar also append bar to PATH Senv PATH wbin prepend wbin to PATH senv d VISUAL forget VISUAL persistent val env p print all persistent env modifications 4 4 5 Prompt customization Here are some prompt configurations you can try out interactively by using the config magic Sconfig PromptManager in_template r color LightGreen u h color LightBlue color Ligh Sconfig PromptManager in2_template r color Green color LightGreen D colon Green gt Sconfig PromptManager out_template r lt gt You can change the prompt configuration to your liking permanently by editing ipython_config py c PromptManager in_template r color LightGreen u h color LightBlue coloy LightCyan c PromptManager in2_template r color Green color LightGreen D color Green gt c PromptManager out_template r lt gt Read more about the configuration system for details on how to find ipython_con
515. t used for starting Python s parallel computing processes as well as the in tegration between Python and the Vision environment http mgltools scripps edu packages vision This project would not have been possible without the support and leadership of Jose Unpingco from Ohio State Tech X Corporation for sponsoring a NASA SBIR project in 2008 on Python s distributed array and parallel computing capabilities 420 Chapter 9 About IPython IPython Documentation Release 1 2 1 An Afternoon Hack e Bivio Software http www bivio biz bp Intro for hosting an Python sprint in 2006 in addition to their support of the Front Range Pythoneers group in Boulder CO 9 1 4 Contributors And last but not least all the kind Python contributors who have contributed new code bug reports fixes comments and ideas A brief list follows please let us know if we have omitted your name by accident Mark Voorhies lt mark voorhies AT ucsf edu gt Printing support in Qt console Justin Riley lt justin t riley AT gmail com gt Contributions to parallel support Amazon EC2 Sun Grid Engine documentation Satrajit Ghosh lt satra AT mit edu gt parallel computing SGE and much more Thomas Spura lt tomspur AT fedoraproject org gt various fixes motivated by Fedora support Omar Andr s Zapata Mesa lt andresete chaos AT gmail com gt Google Summer of Code 2010 terminal support with ZeroMQ Gerardo Gutierrez lt muzgash AT gmail
516. t with title gt This behavior is controlled by the InlineBackend close_figures configurable and if you set it to False via config or config file then Python will not close figures and tools like gcf gca getfigs will behave the same as they do with other backends You will however have to manually close figures close all active figures In 13 fig close for fig in getfigs 4 5 3 Saving and Printing IPythonQt has the ability to save your current session as either HTML or XHTML If you have been using display or inline matplotlib your figures will be PNG in HTML or inlined as SVG in XHTML PNG images have the option to be either in an external folder as in many browsers Webpage Complete option or inlined as well for a larger but more portable file Note Export to SVG XHTML requires that you are using SVG figures which is not the default To switch the inline figure format to use SVG during an active session do In 10 config InlineBackend figure_format svg Or you can add the same line c Inline instead of config Inline to your config files This will only affect figures plotted after making this call The widget also exposes the ability to print directly via the default print shortcut or context menu Note Saving is only available to richtext Qt widgets which are used by default but if you pass the plain flag saving will not be ava
517. talled Expose store_history to execute_request messages Add welcome_message option to enable_pylab Fix variable expansion on self Remove code from prefilter that duplicates functionality in inputsplitter 2 3 Issues closed in the 1 0 development cycle 45 IPython Documentation Release 1 2 1 An Afternoon Hack PR 2295 PR 2280 PR 2290 PR 2288 PR 2287 PR 2218 PR 2277 PR 2045 PR 2271 PR 2278 PR 2225 PR 2246 PR 2274 PR 2270 PR 2269 PR 2263 PR 2256 PR 2255 PR 2261 PR 2260 PR 2242 PR 2250 PR 2245 PR 2248 PR 2238 PR 2211 PR 2236 PR 2231 PR 2232 PR 2235 PR 2222 PR 2224 allow pip install from github repository directly fix SSH passwordless check for OpenSSH nbmanager s assertEquals assertEqual again Removed outdated dev docs Use redirect for new notebooks nb up down arrow keys move to begin end of line at top bottom of cell Refactoring notebook managers and adding Azure backed storage use display instead of send_figure in inline backend hooks allow disabling SQLite history Add annotate option to cython magic serialize individual args kwargs rather than the containers CLN Use name to id mapping of notebooks instead of searching SSHLauncher tweaks add missing location when disambiguating controller IP Allow docs to build on http readthedocs org Adding data publication example notebook better flus
518. tatements Python does not use any of these by default 6 6 Custom input transformation 363 IPython Documentation Release 1 2 1 An Afternoon Hack An InteractiveShell instance actually has two IPythonInputSplitter instances as the attributes input_splitter to tell when a block of input is complete and input_transformer_manager to transform complete cells If you add a transformer you should make sure that it gets added to both Stateless transformations The simplest kind of transformations work one line at a time Write a function which takes a line and returns a line and decorate it with StatelessInputTransformer wrap StatelessInputTransformer wrap def my_special_commands line if line startswith 7 return specialcommand repr line return line ey The decorator returns a factory function which will produce instances of StatelessInputTransformer using your function Coroutine transformers More advanced transformers can be written as coroutines The coroutine will be sent each line in turn followed by None to reset it It can yield lines or None if it is accumulating text to yield at a later point When reset it should give up any code it has accumulated This code in IPython strips a constant amount of leading indentation from each line in a cell CoroutineInputTransformer wrap def leading_indent Remove leading indentation If the first line starts w
519. te Print Loop REPL environ ment by decoupling the evaluation into its own process We call this process a kernel it receives execution instructions from clients and communicates the results back to them This decoupling allows us to have several clients connected to the same kernel and even allows clients and kernels to live on different machines With the exclusion of the traditional single process terminal based Python what you start if you run ipython without any subcommands all other Python machinery uses this two process model This includes ipython console ipython qtconsole and ipython notebook As an example this means that when you start ipython qtconsole you re really starting two pro cesses a kernel and a Qt based client can send commands to and receive results from that kernel If there is already a kernel running that you want to connect to you can pass the existing flag which will skip initiating a new kernel and connect to the most recent kernel instead To connect to a specific kernel once you have several kernels running use the Sconnect_info magic to get the unique connection file which will be something like existing kernel 19732 4son but with different numbers which correspond to the Process ID of the kernel You can read more about using ipython gtconsole and ipython notebook There is also a message spec which documents the protocol for communication between kernels and clients See also Frontend Ker
520. ted quickref to show short hand for sc and sx 1855 script and file magics 1863 Ability to silence a cell in the notebook 1870 add capture for capturing stdout err 1861 Use dvipng to format sympy Matrix 1867 Fix 1px margin bouncing of selected menu item 1889 Reconnect when the websocket connection closes unexpectedly 1577 If a notebook loses its network connection WebSockets won t reconnect 1886 Fix a bug in renaming notebook 1895 Fix error in test suite with ip system 1762 add locate entry points 1883 Fix vertical offset due to bold italics and bad browser fonts 1875 re write columnize with intermediate step 1860 Python utils columnize sometime wrong 1851 new completer for qtconsole 1892 Remove suspicious quotes in interactiveshell py 1854 Class hierarchy and graphiz dot magics 108 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 1741 1827 1864 1829 1832 1674 1874 1882 1878 1879 1840 1841 1736 1850 1663 1872 1869 1781 1815 1819 1838 1839 1782 1862 1859 1857 1856 1848 1852 1780 1853 1842 Sending tracebacks over ZMQ should protect against unicode failure Rmagic exceptions notebook don t care about leading prct in completion Make svg jpeg and png images resizable in notebook HTML Notebook carriage return ha
521. ted the principle of loose coupling 88 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack 2 5 Issues closed in the 0 13 development cycle 2 5 1 Issues closed in 0 13 GitHub stats since Python 0 12 2011 12 19 2012 06 30 These lists are automatically generated and may be incomplete or contain duplicates The following 62 authors contributed 1760 commits Aaron Culich Aaron Meurer Alex Kramer Andrew Giessel Andrew Straw Andr Matos Aron Ahmadia Ben Edwards Benjamin Ragan Kelley Bradley M Froehle Brandon Parsons Brian E Granger Carlos Cordoba David Hirschfeld David Zderic Ernie French Fernando Perez Tan Murray Jason Grout Jens H Nielsen Jez Ng Jonathan March Jonathan Taylor Julian Taylor Jorgen Stenarson Kent Inverarity 2 5 Issues closed in the 0 13 development cycle 89 IPython Documentation Release 1 2 1 An Afternoon Hack Marc Abramowitz Mark Wiebe Matthew Brett Matthias BUSSONNIER Michael Droettboom Mike Hansen Nathan Rice Pankaj Pandey Paul Paul Ivanov Piotr Zolnierczuk Piti Ongmongkolkul Puneeth Chaganti Robert Kern Ross Jones Roy Hyunjin Han Scott Tsai Skipper Seabold Stefan van der Walt Steven Johnson Takafumi Arakaki Ted Wright Thomas Hisch Thomas Kluyver Thomas Spura Thomi Richards Tim Couper Timo Paulssen Toby Gilham Tony S Yu 23 Trevor King Walter Doerwald
522. ternoon Hack PyPy support The terminal interface to Python now runs under PyPy We will continue to monitor PyPy s progress and hopefully before long at least we ll be able to also run the notebook The Qt console may take longer as Qt is a very complex set of bindings to a huge C library and that is currently the area where PyPy still lags most behind But for everyday interactive use at the terminal with this release and PyPy 1 7 things seem to work quite well from our admittedly limited testing Other important new features SSH Tunnels In 0 11 the TPython parallel Client could tunnel its connections to the Con troller via ssh Now the QtConsole supports ssh tunneling as do parallel engines relaxed command line parsing 0 11 was released with overly strict command line parsing pre venting the ability to specify arguments with spaces e g ipython pylab qt or ipython c print hi This has been fixed by using argparse The new parsing is a strict superset of 0 11 so any commands in 0 11 should still work in 0 12 HistoryAccessor The HistoryManager class for interacting with your Python SQLite history database has been split adding a parent HistoryAccessor class so that users can write code to access and search their Python history without being in an Python session PR 824 kernel gui and pylab The gui and pylab magics have been restored to the Python kernel e g in the qtconsole or notebook
523. tests 142 Yreset doesn t reset prompt number Make ip reset work the same in interactive or non interactive code 3 make readline a dependency on OSX restore auto debug behavior fix for issue 337 incorrect phantom tooltips for magics newparallel branch add zmq parallel submodule Hard reset Unicode win process AST splitter Removetwisted Magic pastebin Bug tests for GH Issues 238 284 306 307 Skip module machinery if not installed Known failures reported as K 331 319 329 179 Tweak config loader for PyPy compatibility Rewrite code to restore readline history after an action Do not store file contents in history when running a ipy file Html notebook 3 Add missing external pexpect to packages Magic local scope Unicode magic args allow Unicode Command Line options Readline shortcuts 170 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e 311 e 312 e 294 e 292 e 252 239 e 236 e 261 e 282 e 191 199 e 204 293 249 283 e 281 237 e 251 e 278 e 275 280 259 193 272 e 219 e 221 e 230 224 229 e 273 228 233 Qtconsole exit History memory Issue 290 Issue 31 Unicode issues Fix history magic command s bugs wrt to full history and add O option to display full history History minus p flag Adapt magic commands to new hist
524. text cell gt raw cell Fix Ynotebook magic etc nbformat unicode tests and fixes Gtk3 integration with ipython works Examples syntax avoid errors installing on Python 3 Find encoding for Python files Fix writing git commit ID to a file on build with Python 3 98 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 1556 PR 1502 PR 1445 PR 1538 PR 1546 PR 1559 PR 1563 PR 1560 PR 1569 PR 1573 PR 1568 PR 1567 PR 1565 PR 1552 PR 1541 PR 1544 PR 1517 PR 1519 PR 1489 PR 1477 PR 1514 PR 1481 PR 1496 PR 1508 PR 1495 PR 1472 PR 1483 PR 1383 PR 1398 PR 1440 PR 1419 PR 1424 shallow copy DictDB query results small changes in response to pyflakes pass Don t build sphinx docs for sdists store git commit hash in utils _sysinfo instead of hidden data file attempt to suppress exceptions in channel threads at shutdown update tools github_stats py to use GitHub API v3 clear_output improvements remove obsolete discussion of Twisted trial from testing docs BUG qtconsole non standard handling of a and b Fixes 1561 BUG Ctrl C crashes wx pylab kernel in qtconsole fix PR 1567 Fix openssh_tunnel did not parse port in server fix AsyncResult abort use os getcwdu in NotebookManager display_pub flushes stdout err make MultiKernelManager kernel_manager_class configurable Fix indentation bug in IP
525. the Python format as a way to output a script version of a notebook and the import capabilities as a way to load existing code to get a notebook started But the Python version is not an alternate notebook format 4 6 The IPython Notebook 251 IPython Documentation Release 1 2 1 An Afternoon Hack See also Notebook JSON file format 4 7 Converting notebooks to other formats Newly added in the 1 0 release of Python is the nbconvert tool which allows you to convert an ipynb notebook document file into various static formats Currently noconvert is provided as a command line tool run as a script using Python A direct export capability from within the Python Notebook web app is planned The command line syntax to run the nbconvert script is ipython nbconvert to FORMAT notebook ipynb This will convert the Python document file notebook ipynb into the output format given by the FORMAT string The default output format is html for which the to argument may be omitted ipython nbconvert notebook ipynb Python provides a few templates for some output formats and these can be specified via an additional template argument The currently supported export formats are s to html template full default A full static HTML render of the notebook This looks very similar to the interactive view template basic Simplified HTML useful for embedding in webpage
526. the embedded Python mode and without Python at all For an embedded shell see sec Embedding simply call the constructor with pdb in the argument string and pdb will automatically be called if an uncaught exception is triggered by your code For stand alone use of the feature in your programs which do not use Python at all put the following lines toward the top of your main routine import sys from IPython core import ultratb sys excepthook ultratb FormattedTB mode Verbose color_scheme Linux call_pdb 1 4 3 IPython reference 225 IPython Documentation Release 1 2 1 An Afternoon Hack The mode keyword can be either Verbose or Plain giving either very detailed or normal tracebacks respectively The color_scheme keyword can be one of NoColor Linux default or LightBG These are the same options which can be set in Python with colors and xmode This will give any of your programs detailed colored tracebacks with automatic invocation of pdb 4 3 6 Extensions for syntax processing This isn t for the faint of heart because the potential for breaking things is quite high But it can be a very powerful and useful feature In a nutshell you can redefine the way Python processes the user input line to accept new special extensions to the syntax without needing to change any of Python s own code In the Python extensions directory you will find some exa
527. the notebook and access it later by pointing your browser https your host com 9999 with ipython notebook profile nbserver 256 Chapter 4 Using IPython for interactive work ycert pem IPython Documentation Release 1 2 1 An Afternoon Hack 4 8 3 Running with a different URL prefix The notebook dashboard the landing page with an overview of the notebooks in your working direc tory typically lives at the URL http localhost 8888 If you prefer that it lives together with the rest of the notebook under a sub directory e g http localhost 8888 ipython you can do so with configuration options like the following see above for instructions about modifying ipython_notebook_config py c NotebookApp base_project_url ipython c NotebookApp base_kernel_url ipython c NotebookApp webapp_settings static_url_prefix ipython static 4 8 4 Using a different notebook store By default the notebook server stores the notebook documents that it saves as files in the working di rectory of the notebook server also known as the notebook_dir This logic is implemented in the FileNotebookManager class However the server can be configured to use a different notebook man ager class which can store the notebooks in a different format Currently we ship a AzureNotebookManager class that stores notebooks in Azure blob storage This can be used by adding the following lines to your ipython_notebook_config py file
528. the qtconsole while leaving a visible indication that the kernel has restarted PR 1681 e Allow the native display of jpeg images in the qtconsole PR 1643 Parallel The parallel tools have been improved and fine tuned on multiple fronts Now the creation of an IPython parallel Client object automatically activates a line and cell magic function px that sends its code to all the engines Further magics can be easily created with the Client activate method to conveniently execute code on any subset of engines PR 1893 The px cell magic can also be given an optional targets argument as well as a out argument for storing its output A new magic has also been added 3pxconfig that lets you configure various defaults of the parallel magics As usual type 3pxconfig for details The exception reporting in parallel contexts has been improved to be easier to read Now Python directly reports the remote exceptions without showing any of the internal execution parts In 1 from IPython parallel import Client c Client In 2 px 1 0 execute ZeroDivisionError Traceback most recent call last lt ipython input 1 5c9758a9c21 gt in lt module gt gt 1 1 0 ZeroDivisionError integer division or modulo by zero 1 execute ZeroDivisionError Traceback most recent call last lt ipython input 1 5c9758a9c21 gt in lt module gt gt 11 8 ZeroDivisionError integer division or modulo by zero
529. ther MPIExec launcher names are now prefixed with just MPI to better match other batch launch ers and be generally more intuitive The MPIExec names are deprecated but continue to work For embedding a shell note that the parameters user_global_ns and global_ns have been deprectated in favour of user_module and module respsectively The new parameters expect a module like object rather than a namespace dict The old parameters remain for backwards compati bility although user_global_ns is now ignored The user_ns parameter works the same way as before and calling embed with no arguments still works as before Development summary and credits The previous version IPython 0 11 was released on July 31 2011 so this release cycle was roughly 4 1 2 months long we closed a total of 515 issues 257 pull requests and 258 regular issues a detailed list is available Many users and developers contributed code features bug reports and ideas to this release Please do not hesitate in contacting us if we ve failed to acknowledge your contribution here In particular for this release we have had commits from the following 45 contributors a mix of new and regular names in alphabetical order by first name Alcides lt alcides at do not span me com gt Ben Edwards lt bedwards at cs unm edu gt Benjamin Ragan Kelley lt benjaminrk at gmail com gt Benjamin Thyreau lt benjamin thyreau at gmail com gt Bernardo B Marques lt bernardo fir
530. these magics you must register them with a running IPython This code must be placed in a file that is loaded once IPython is up and running ip get_ipython You can register the class itself without instantiating it JIPython will call the default constructor on it ip register_magics MyMagics If you want to create a class with a different constructor that holds additional state then you should always call the parent constructor and instantiate the class yourself before registration magics_class class StatefulMagics Magics Magics that hold additional state def init__ self shell data You must call the parent constructor super StatefulMagics self __init__ shell self data data etc This class must then be registered with a manually created instance since its constructor has different arguments from the default ip get_ipython magics StatefulMagics ip some_data ip register_magics magics In earlier versions Python had an API for the creation of line magics cell magics did not exist at the time that required you to create functions with a method looking signature and to manually pass both the function and the name While this API is no longer recommended it remains indefinitely supported for backwards compatibility purposes With the old API you d create a magic as follows def func self line print Line magic called with line line print TIP
531. this kernel use IPKernelApp existing kernel 12345 json Which we can connect to from the client with user client gt ipython qtconsole ssh login ip 192 168 1 123 existing pat Note The IP here is the address of worker as seen from login and need only be specified if the kernel used the ambiguous 0 0 0 0 all interfaces address If it had used 192 168 1 123 to start with it would not be needed Manual SSH tunnels It s possible that Python s ssh helper functions won t work for you for various reasons You can still connect to remote machines as long as you set up the tunnels yourself The basic format of forwarding a local port to a remote one is client gt ssh lt server gt lt localport gt lt remoteip gt lt remoteport gt f N This will forward local connections to localport on client to remoteip remoteport via server Note that remoteip is interpreted relative to server not the client So if you have direct ssh access to the machine to which you want to forward connections then the server is the remote machine and remoteip should be 242 Chapter 4 Using IPython for interactive work h to kernel IPython Documentation Release 1 2 1 An Afternoon Hack server s IP as seen from the server itself i e 127 0 0 1 Thus to forward local port 12345 to remote port 54321 on a machine you can see do client gt ssh machine 12345 127 0 0 1 54
532. thon as we know it today grew out of the following three projects ipython by Fernando P rez Fernando began using Python and ipython began as an outgrowth of his desire for things like Mathematica style prompts access to previous output again like Mathematica s syntax and a flexible configuration system something better than PYTHONSTARTUP IPP by Janko Hauser Very well organized great usability Had an old help system IPP was used as the container code into which Fernando added the functionality from ipython and LazyPython LazyPython by Nathan Gray Simple but very powerful The quick syntax auto parens auto quotes and verbose colored tracebacks were all taken from here Here is how Fernando describes the early history of IPython When I found out about IPP and LazyPython I tried to join all three into a unified system I thought this could provide a very nice working environment both for regular programming 9 2 History 423 IPython Documentation Release 1 2 1 An Afternoon Hack and scientific computing shell like features IDL Matlab numerics Mathematica type prompt history and great object introspection and help facilities I think it worked reasonably well though it was a lot more work than I had initially planned 9 3 License and Copyright 9 3 1 License Python is licensed under the terms of the new or revised BSD license as follows Copyright c 2011 IPython Development Team All
533. tion 5 1 2 Introduction This section gives an overview of Python s sophisticated and powerful architecture for parallel and dis tributed computing This architecture abstracts out parallelism in a very general way which enables Python to support many different styles of parallelism including e Single program multiple data SPMD parallelism e Multiple program multiple data MPMD parallelism e Message passing using MPI e Task farming e Data parallel e Combinations of these approaches e Custom user defined approaches Most importantly Python enables all types of parallel applications to be developed executed debugged and monitored interactively Hence the I in IPython The following are some example usage cases for Python e Quickly parallelize algorithms that are embarrassingly parallel using a number of simple approaches Many simple things can be parallelized interactively in one or two lines of code e Steer traditional MPI applications on a supercomputer from an Python session on your laptop e Analyze and visualize large datasets that could be remote and or distributed interactively using Python and tools like matplotlib TVTK 259 IPython Documentation Release 1 2 1 An Afternoon Hack Develop test and debug new parallel algorithms that may use MPI interactively Tie together multiple MPI jobs running on different systems into one giant distributed and parallel system Start a par
534. tion without having to go to the bottom with Shift Enter 4 5 1 load The new 1load magic previously 1oadpy takes any script and pastes its contents as your next input so you can edit it before executing The script may be on your machine but you can also specify an history range or a url and it will download the script from the web This is particularly useful for playing with examples from documentation such as matplotlib In 6 load http matplotlib org plot_directive mpl_examples mplot3d contour3d_demo py In 7 from mpl_toolkits mplot3d import axes3d import matplotlib pyplot as plt fig plt figure ax fig add_subplot 111 projection 3d X Y Z axes3d get_test_data 0 05 cset ax contour X Y ax clabel cset fontsize 9 inline 1 plt show 234 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack 4 5 2 Inline Matplotlib One of the most exciting features of the QtConsole is embedded matplotlib figures You can use any standard matplotlib GUI backend to draw the figures and since there is now a two process model there is no longer a conflict between user input and the drawing eventloop In 34 from scipy special import jn In 35 x Linspace 4 pi In 36 for i in range 6 plot x jn i x 1 0 0 5 display Python provides a function display for displaying rich
535. tion and traceback coloring support For even easier debugger access try debug after seeing an exception e Profiler support You can run single statements similar to profile run or complete programs under the profiler s control While this is possible with standard cProfile or profile modules Python wraps this functionality with magic commands see prun and run p convenient for rapid interactive work e Simple timing information You can use the 3t imeit command to get the execution time of a Python statement or expression This machinery is intelligent enough to do more repetitions for commands that finish very quickly in order to get a better estimate of their running time In 1 Stimeit 1 1 10000000 loops best of 3 25 5 ns per loop In 2 Stimeit math sin x for x in range 5000 1000 loops best of 3 719 us per loop To get the timing information for more than one expression use the 3Stimeit cell magic command e Doctest support The special doctest_mode command toggles a mode to use doctest compatible prompts so you can use Python sessions as doctest code By default Python also allows you to paste existing doctests and strips out the leading gt gt gt and prompts in them 1 2 Enhanced interactive Python shell 3 IPython Documentation Release 1 2 1 An Afternoon Hack 1 3 Decoupled two process model IPython has abstracted and extended the notion of a traditional Read Evalua
536. tiveShellEmbed Now create an instance of the embeddable shell The first argument is a string with options exactly as you would type them if you were starting IPython at the system command line Any parameters you want to define for configuration can thus be specified here ipshell InteractiveShellEmbed config cfg bannerl Dropping into IPython exit_msg Leaving Interpreter back to program Sb Se SR SR Make a second instance cfg2 cfg copy prompt_config cfg2 PromptManager you can have as many as you want prompt_config in_template In2 lt gt if not nested prompt_config in_template In2 lt gt prompt_config in2_template eNNDes t prompt_config out_template Out lt gt ipshell2 InteractiveShellEmbed config cfg bannerl Second IPython instance print nHello This is printed from the main controller program n You can then call ipshell anywhere you need it with an optional message ipshell xx xCalled from top level Hit Ctrl D to exit interpreter and continue program n Note that if you use kill_embedded you can fully deactivate n This embedded instance so it will never turn on again print nBack in caller program moving along n More details InteractiveShellEmbed instances don t print the standard system banner and messages The IPython banner which actually may contain initialization messages is ava
537. to the username for the process that generated the message This is useful in collaborative settings where multiple users may be interacting with the same kernel simultaneously so that frontends can label the various messages in a meaningful way header msg_id uuid username str session uuid All recognized message type strings are listed below msg_type str In a chain of messages the header from the parent is copied so that clients can track where messages come from parent_header dict 7 2 Messaging in IPython 385 IPython Documentation Release 1 2 1 An Afternoon Hack Any metadata associated with the message metadata dict The actual content of the message must be a dict whose structure depends on the message typ content dict 7 2 3 The Wire Protocol This message format exists at a high level but does not describe the actual implementation at the wire level in zeromq The canonical implementation of the message spec is our Session class Note This section should only be relevant to non Python consumers of the protocol Python consumers should simply import and use IPython s own implementation of the wire protocol in the TPython kernel zmq session Session object Every message is serialized to a sequence of at least six blobs of bytes bb u U i d b lt IDS MSG gt b baddad42 b header b
538. traitlets module which provides the Unicode Int and Float types In addition to these traitlets the Python utils traitlets provides traitlets for a number of other types Note Underneath the hood the Configurable base class is a subclass of IPython utils traitlets HasTraits The IPython utils traitlets module is a lightweight version of enthought traits Our implementation is a pure Python subset mostly API compatible of enthought traits that does not have any of the automatic GUI generation capabilities Our plan is to achieve 100 API compatibility to enable the actual enthought t raits to eventually be used instead Currently we cannot use enthought traits as we are committed to the core of Python being pure Python It should be very clear at this point what the naming convention is for configuration attributes c ClassName attribute_name attribute_value Here ClassName is the name of the class whose configuration attribute you want to set attribute_name is the name of the attribute you want to set and att ribute_value the the value you want it to have The ClassName attribute of c is not the actual class but instead is another Config instance Note The careful reader may wonder how the ClassName MyClass in the above example attribute of the configuration object c gets created These attributes are created on the fly by the Config instance using a simple naming convention Any attribute of a Config
539. trieval across sessions and its own special command system for adding functionality when working interactively It tries to be a very efficient environment both for Python code development and for exploration of problems using Python objects in situations like data analysis 2 Serve as an embeddable ready to use interpreter for your own programs An interactive Python shell can be started with a single call from inside another program providing access to the current namespace This can be very useful both for debugging purposes and for situations where a blend of batch processing and interactive exploration are needed 3 Offer a flexible framework which can be used as the base environment for working with other systems with Python as the underlying bridge language Specifically scientific environments like Mathematica IDL and Matlab inspired its design but similar ideas can be useful in many fields IPython Documentation Release 1 2 1 An Afternoon Hack 4 Allow interactive testing of threaded graphical toolkits Python has support for interactive non blocking control of GTK Qt WX GLUT and OS X applications via special threading flags The normal Python shell can only do this for Tkinter applications 1 2 1 Main features of the interactive shell Dynamic object introspection One can access docstrings function definition prototypes source code source files and other details of any object accessible to the interpreter
540. ts 71 temp file errors in iptest Python core 350 Decide how to handle displayhook being triggered multiple times 360 Remove runlines method 125 Exec lines in config should not contribute to line numbering or history 20 Robust readline support on OS X s builtin Python 147 On Windows page is being too restrictive to split line by rn only 326 Update docs and examples for parallel stuff to reflect movement away from Twisted 341 FIx Parallel Magics for newparallel 338 Usability improvements to Qt console 142 unexpected auto indenting when varibles names that start with pass 2 9 Issues closed in the 0 11 development cycle 177 IPython Documentation Release 1 2 1 An Afternoon Hack 296 Automatic PDB via pdb doesn t work 337 exit and quit in Qt console produces phantom signature docstring popup even though quit or exit raises NameError 318 debug broken in master invokes missing save_history method 307 lines ending with semicolon should not go to cache 104 have ipengine run start up scripts before registering with the controller 33 The skip_doctest decorator is failing to work on Shell MatplotlibShellBase magic_run 336 Missing figure development figs iopubfade png for docs 49 clear should also delete _NN references and Out NN ones 335 using setuptools installs every script twice 306 multiline strings at end of input cause noop 327 PyPy compatibility 328 run script ipy
541. ts Run your doctests from within Python for development and debugging The special doctest_mode com mand toggles a mode where the prompt output and exceptions display matches as closely as possible that of the default Python interpreter In addition this mode allows you to directly paste in code that contains leading gt gt gt prompts even if they have extra leading whitespace as is common in doctest files This combined with the history t call to see your translated history allows for an easy doctest workflow where you can go from doctest to interactive execution to pasting into valid Python code as needed 4 2 3 Use IPython to present interactive demos Use the IPython 1lib demo Demo class to load any Python script as an interactive demo With a minimal amount of simple markup you can control the execution of the script stopping as needed See here for more 206 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack 4 2 4 Suppress output Put a at the end of a line to suppress the printing of output This is useful when doing calculations which generate long output you are not interested in seeing It also keeps the object out of the output cache so if you re working with large temporary objects they ll be released from memory sooner 4 2 5 Lightweight version control When you call edit with no arguments Python opens an empty editor with a tempor
542. ts This remains but due to the addition of fully non copying sends of arrays and buffers there is also a track flag which instructs PyZMQ to produce a MessageTracker that will let you know when it is safe again to edit arrays in place The result of a non blocking call to apply is now an AsyncResult_ object described below 5 14 4 MultiEngine to DirectView The multiplexing interface previously provided by the MultiEngineClient is now provided by the Di rectView Once you have a Client connected you can create a DirectView with index access to the client view client 1 5 The core methods for communicating with engines remain execute run push pull scatter gather These methods all behave in much the same way as they did on a Multi EngineClient old In 2 mec execute a 5 targets 0 1 2 new In 2 view execute a 5 targets 0 1 2 or In 2 rce 0 1 2 execute a 5 This extends to any method that communicates with the engines Requests of the Hub queue status etc are no longer asynchronous and do not take a block argument e get_ids is now the property ids which is passively updated by the Hub no need for network requests for an up to date list e barrier has been renamed to wait and now takes an optional timeout flush is re moved as it is redundant with wait e zip_pull has been removed 5 14 Transitioning from IPython kernel to IPython parallel 34
543. tup files not executed with notebook 1197 Interactive shell trying to from import history 1198 Kernel Has Died error in Notebook 1201 env magic fails with Python 3 2 1204 double newline from loadpy in python notebook at least on mac 1208 should dv sync_import print failed imports 1225 SyntaxError display broken in Python 3 2 7 Issues closed in the 0 12 development cycle 135 IPython Documentation Release 1 2 1 An Afternoon Hack 1232 1241 1254 1260 1268 1269 1281 1302 1304 1317 1345 1375 1401 1408 1412 1421 1484 Dead kernel loop When our debugger class is used standalone _oh key errors are thrown typo in notebooklist js breaks links heartbeat failure on long gil holding operation notebook reset magic fails with StdinNotImplementedError Another strange input handling error in Hub registration_timeout must be an integer but heartmonitor period is CFloat Input parsing with prun clobbers escapes controller server load can disrupt heartbeat Very slow traceback construction from Cython extension notebook can t save unicode as script history g f file encoding issue numpy arrays cannot be used with View apply in Python 3 test_get_home_dir_3 failed on Mac OS X Input parsing issue with oprun ipython32 run d breaks with NameError name execfile is not defined unhide git_commit_info ini 2 2 Issues closed in 0 12 In th
544. ub has an excellent series of tutorials The pro git book is a good in depth book on git A git cheat sheet is a page giving summaries of common commands The git user manual The git tutorial The git community book git ready a nice series of tutorials git casts video snippets giving git how tos git magic extended introduction with intermediate detail Fernando Perez git page Fernando s git page many links and tips A good but technical page on git concepts Th git parable is an easy read explaining the concepts behind git git svn crash course git for those of us used to subversion Advanced git workflow There are many ways of working with git here are some posts on the rules of thumb that other projects have come up with e Linus Torvalds on git management e Linus Torvalds on linux git workflow Summary use the git tools to make the history of your edits as clean as possible merge from upstream edits as little as possible in branches where you are doing active development Manual pages online You can get these on your own machine with e g git help push or same thing git push help but for convenience here are the online manual pages for some common commands git add git branch git checkout git clone 382 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack git commit git config git diff git log git pull git push
545. uffers are added on to the end of the message and can be any objects that present the buffer interface 7 4 Connection Diagrams of The IPython ZMQ Cluster This is a quick summary and illustration of the connections involved in the ZeroMQ based Python cluster for parallel computing 7 4 1 All Connections The Python cluster consists of a Controller and one or more each of clients and engines The goal of the Controller is to manage and monitor the connections and communications between the clients and the engines The Controller is no longer a single process entity but rather a collection of processes specifically one Hub and 4 or more Schedulers It is important for security practicality reasons that all connections be inbound to the controller processes The arrows in the figures indicate the direction of the connection The Controller consists of 1 5 processes Central to the cluster is the Hub which monitors engine state execution traffic and handles registration and notification The Hub includes a Heartbeat Monitor for keep ing track of engines that are alive Outside the Hub are 4 Schedulers These devices are very small pure C MonitoredQueue processes or optionally threads that relay messages very fast but also send a copy of each message along a side socket to the Hub The MUX queue and Control queue are MonitoredQueue MQ devices which relay explicitly addressed messages from clients to engines and their replies back up
546. ught Python Distribution EPD http www enthought com products epd php EPD is produced by Enthought Inc and contains all of these packages and others in a single installer and is available free for academic users While it is also possi ble to download and install each package individually this is a tedious process Thus we highly recommend using EPD to install these packages on Windows Regardless of how you install the dependencies here are the steps you will need to follow 1 Install all of the packages listed above either individually or using EPD on the head node compute nodes and user workstations 2 Make sure that C Python27 and C Python27 Scripts are in the system PATH variable on each node 3 Install the latest development version of IPython This can be done by downloading the the develop ment version from the Python website http ipython org and following the installation instructions Further details about installing Python or its dependencies can be found in the online Python documenta tion http ipython org documentation html Once you are finished with the installation you can try Python out by opening a Windows Command Prompt and typing ipython This will start Python s interactive shell and you should see something like the following 316 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack Microsoft Windows Version 6 0 6001
547. ul new feature contributed by Robert Kern IPython demos created with the IPython demo module can now be created from files on disk or strings in memory Other fixes and improvements to the demo system by Tom Fetherston Added find_cmd function to I Python platutils module to find commands in a cross platform manner Many improvements and fixes to Ga l Varoquaux s ipythonx a WX based lightweight Python instance that can be easily embedded in other WX applications These improvements have made it possible to now have an embedded Python in Mayavi and other tools MultiengineClient objects now have a benchmark method The manual now includes a full set of auto generated API documents from the code sources using Sphinx and some of our own support code We are now using the Numpy Documentation Standard for all docstrings and we have tried to update as many existing ones as possible to this format The new IPython Extensions ipy_pretty extension by Robert Kern provides configurable pretty printing Many improvements to the ipython wx standalone WX based Python application by Laurent Dufr chou It can optionally run in a thread and this can be toggled at runtime allowing the loading of Matplotlib in a running session without ill effects Python includes a copy of Steven Bethard s argparse in the IPython external package so we can use it internally and it is also available to any Python user By installing it i
548. up remove numpy install from travis tox scripts New keybinding for code cell execution cell insertion Updating the parallel options pricing example expand line in cell magics Fix tab completion with Python embed_kernel embed Default to the future compiler flags of the calling frame fix remote_profie_dir typo in SSH launchers 2to3 compat Tuple params in func defs Fix unittest Deprecation Warnings Refactor test_pr py 2to3 Apply has_key fixer Add option append a to save use explicit url in notebook example Tell git that py files contain Python code for use in word diffs Apply 2to3 next fix ipcluster broken with any batch launcher PBS LSF SGE Windows make file for Sphinx documentation Make BG color of inline plot configurable BUG Look up the _repr_pretty_ method on the class within the MRO rath in progress python 2 and 3 compatibility without 2to3 second try open notebook copy in different tabs allows password and prefix for notebook Print View re aliad ed to edit in qtconsole Fixes and improvements to the input splitter fix completer deletting newline Fix logging on interactive shell Fix some Python 3 2 Resource Warnings conform to pep 3110 Skip notebook static dir in test suite 48 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack Issues 691 PR 2063 Remove umlauts so py3 installations on LANG C systems succee
549. ures it provides Integration into Microsoft Visual Studio Thanks to the work of the Microsoft Python Tools for Visual Studio team this version of Python has been integrated into Microsoft Visual Studio s Python tools open source plug in Details below Improved unicode support We closed many bugs related to unicode input Python 3 Python now runs on Python 3 x See Python 3 support for details New profile model Profiles are now directories that contain all relevant information for that session and thus better isolate Python use cases SQLite storage for history All history is now stored in a SQLite database providing support for multiple simultaneous sessions that won t clobber each other as well as the ability to perform queries on all stored data New configuration system All parts of Python are now configured via a mechanism inspired by the Enthought Traits library Any configurable element can have its attributes set either via files that now use real Python syntax or from the command line Pasting of code with prompts Python now intelligently strips out input prompts be they plain Python ones gt gt gt and or Python ones In N and More details here hors and support Over 60 separate authors have contributed to this release see below for a full list In particular we want to highlight the extremely active participation of two new core team members Evan Patterson implemented the Qt console a
550. used with s and echo add support to automatic retry of tasks add support to continue tasks IPython should display things unsorted if it can t sort them 176 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e 416 wrong encode when printing unicode string e 376 Failing InputsplitterTest e 405 TraitError in traitlets py 332 on any input e 392 UnicodeEncodeError on start e 137 sys getfilesystemencoding return value not checked e 300 Users should be able to manage kernels and kernel sessions from the notebook UI e 301 Users should have access to working Kernel Tabs Edit Help menus in the notebook e 396 cursor move triggers a lot of IO access e 379 Minor doc nit paging argument e 399 Add task queue limit in engine when load balancing 78 StringTask won t take unicode code strings 391 MongoDB add_record does not work in 0 1 1dev 365 newparallel on Windows 386 FAIL test that pushed functions have access to globals 387 Interactively defined functions can t access user namespace 118 Snow Leopard ipy_vimserver POLL error 394 System escape interpreted in multi line string 26 find_job_cmd is too hasty to fail on Windows 368 Installation instructions in dev docs are completely wrong 380 qtconsole pager RST HTML not happening consistently 367 Qt console doesn t support ibus input method 375 Missing libraries cause ImportError in tes
551. uses crash Celltoolbar example issue IPython no longer handles unicode file names Nbconvert windows Inconsistent line endings in markdown cells exported to latex nbconvert add extra blank line to code block on Windows remove spurious print statement from parallel annoted functions Notebook merging a heading cell and markdown cell cannot be undone 2 3 2 Issues closed in 1 1 GitHub stats for 2013 08 08 2013 09 09 since 1 0 These lists are automatically generated and may be incomplete or contain duplicates The following 25 authors contributed 337 commits e Benjamin Ragan Kelley 2 3 Issues closed in the 1 0 development cycle 15 IPython Documentation Release 1 2 1 An Afternoon Hack Bing Xia Bradley M Froehle Brian E Granger Damian Avila dhirschfeld Drazen Lu anin gmbecker Jake Vanderplas Jason Grout Jonathan Frederic Kevin Burke Kyle Kelley Matt Henderson Matthew Brett Matthias Bussonnier Pankaj Pandey Paul Ivanov rossant Samuel Ainsworth Stephan Rave stonebig Thomas Kluyver Yaroslav Halchenko Zachary Sailer We closed a total of 76 issues 58 pull requests and 18 regular issues this is the full list generated with the script tools github_stats py Pull Requests 58 PR 4188 Allow user_ns trait to be None PR 4189 always fire LOCAL_IPS extend PUBLIC_IPS PR 4174 various issues in markdown and rst templates PR 4178 add missing data_javascript PR 4181 nbconvert
552. ut 1 79 lt matplotlib lines Line2D at 0x10b9476d0 gt Out 2 79 lt matplotlib lines Line2D at 0x110652750 gt Out 3 79 lt matplotlib lines Line2D at 0x10c6566d0 gt px Cell Magic px can be used as a Cell Magic which accepts some arguments for controlling the execution Targets and Blocking px accepts targets for controlling which engines on which to run and no block for speci fying the blocking behavior of this cell independent of the defaults for the View In 6 px targets 2 print I am even Parallel execution on engine s 0 2 stdout 0 I am even stdout 2 I am even 290 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack In 7 px targets 1 print I am number 1 Parallel execution on engine s 1 I am number 1 In 8 px print still tallit by default Parallel execution on engine s all stdout 0 still all by default stdout 1 still all by default stdout 2 still all by default stdout 3 still all by default In 9 px noblock import time time sleep 1 time time Async parallel execution on engine s all Out 9 lt AsyncResult execute gt In 0 Spxresult 1339454561 069116 1339454561 076752 1339454561 072837 1339454561 066665 t t t t See also pxconfig accepts these same arguments for chang
553. ut and output history are kept in variables called In and Out keyed by the prompt numbers e g In 4 The last three objects in output history are also kept in variables named _ __ and You can use the history magic function to examine past input and output Input history from previous sessions is saved in a database and Python can be configured to save output history Several other magic functions can use your input history including edit rerun Srecall macro Ssave and pastebin You can use a standard format to refer to lines Spastebin 3 18 20 1 1 5 This will take line 3 and lines 18 to 20 from the current session and lines 1 5 from the previous session 4 1 6 System shell commands To run any command at the system shell simply prefix it with e g ping www bbc co uk You can capture the output into a Python list e g files 1s To pass the values of Python variables or expressions to system commands prefix them with grep rF pattern ipython See our shell section for more details Define your own system aliases It s convenient to have aliases to the system commands you use most often This allows you to work seamlessly from inside Python with the same commands you are used to in your system shell Python comes with some pre defined aliases and a complete system for changing directories both via a stack see pushd popd and dhist and via direct cd The latter keeps a history of
554. ut in dummy mode where calls to it silently return This allows you for example to globally turn off debugging for a program with a single call ipshell dummy_mode True print nTrying to call IPython which is now dummy ipshell print Nothing happened The global dummy mode can still be overridden for a single call print nOverriding dummy mode manually ipshell dummy False Reactivate the IPython shell ipshell dummy_mode False 4 3 IPython reference 223 IPython Documentation Release 1 2 1 An Afternoon Hack print You can even have multiple embedded instances ipshell2 print nMain program calling bar spam n bar spam print Main program finished Bye BAKKKKKKKKKKKKKKKKKKKKe End Of file lt example embed py gt xx KKH KK AK KKK KEK KK KK Once you understand how the system functions you can use the following code fragments in your programs which are ready for cut and paste OQuick code snippets for embedding IPython into other programs See example embed py for full details this file has the bare minimum code for cut and paste use once you understand how to use the system This code loads IPython but modifies a few things if it detects it s running embedded in another IPython session helps avoid confusion try get_ipython except NameError banner exit_msg else banner Nested i
555. ut parentheses or quotes Cell magics are prefixed with a double and they are functions that get as an argument not only the rest of the line but also the lines below it in a separate argument The following examples show how to call the builtin t imeit magic both in line and cell mode In 1 Stimeit range 1000 100000 loops best of 3 7 76 us per loop In 2 S timeit x range 10000 max x 1000 loops best of 3 223 us per loop The builtin magics include Functions that work with code Srun edit save S3macro recall etc e Functions which affect the shell colors xmode Sautoindent Sautomagic etc e Other functions such as reset timeit file load or paste You can always call them using the prefix and if you re calling a line magic on a line by itself you can omit even that run thescript py You can toggle this behavior by running the Sautomagic magic Cell magics must always have the prefix A more detailed explanation of the magic system can be obtained by calling smagic and for more details on any magic function call somemagic to read its docstring To see all the available magic functions call 1 smagic See also Cell magics example notebook Running and Editing The Srun magic command allows you to run any python script and load all of its data directly into the interactive namespace Since the file is re read from disk each time changes you make
556. utdown the kernel If a kernel receives a shutdown request then it aborts all queued messages replies to the request and exits Message type shutdown_request content Message type shutdown_reply content status rok ok or error other error info here as in other messages 7 3 Messaging for Parallel Computing 407 IPython Documentation Release 1 2 1 An Afternoon Hack 7 3 5 Implementation There are a few differences in implementation between the StreamSession object used in the newparallel branch and the Session object the main one being that messages are sent in parts rather than as a single serialized object StreamSession objects also take pack unpack functions which are to be used when serial izing deserializing objects These can be any functions that translate to from formats that ZMQ sockets can send buffers bytes etc Split Sends Previously messages were bundled as a single json object and one call to socket send_json Since the hub inspects all messages and doesn t need to see the content of the messages which can be large messages are now Serialized and sent in pieces All messages are sent in at least 4 parts the header the parent header the metadata and the content This allows the controller to unpack and inspect the always small header without spending time unpacking the content unless the message is bound for the controller B
557. uted in single mode e If there is more than one block if the last one is a single line long run all but the last in exec mode and the very last one in single mode This makes it easy to type simple expressions at the end to see computed values if the last one is no more than two lines long run all but the last in exec mode and the very last one in single mode This makes it easy to type simple expressions at the end to see computed values otherwise last one is also multiline run all in exec mode otherwise last one is also multiline run all in exec mode as a single unit Any error in retrieving the user_variables or evaluating the user_expressions will result in a simple error message in the return fields of the form ERROR ExceptionType Exception message The user can simply send the same variable name or expression for evaluation to see a regular traceback Errors in any registered post_execute functions are also reported similarly and the failing function is re moved from the post_execution set so that it does not continue triggering failures Upon completion of the execution request the kernel always sends a reply with a status code indicating what happened and additional data depending on the outcome See below for the possible return codes and associated data Execution counter old prompt number The kernel has a single monotonically incr
558. value whatever it wants The default behavior of sys displayhook in the Python interactive prompt is to print to sys stdout the repr of the value as long as it is not None which isn t printed at all In our case the kernel instantiates as sys displayhook an object which has similar behavior but which instead of printing to stdout broadcasts these values as pyout messages for clients to display appropriately IPython s displayhook can handle multiple simultaneous formats depending on its configuration The default pretty printed repr text is always given with the data entry in this message Any other formats are provided in the ext ra_formats list Frontends are free to display any or all of these according to its capabilities extra_formats list contains 3 tuples of an ID string a type string and the data The ID is unique to the formatter implementation that created the data Frontends will typically ignore the ID unless if it has requested a particular formatter The type string tells the frontend how to interpret the data It is often but not always a MIME type Frontends should ignore types that it does not understand The data itself is any JSON object and depends on the format It is often but not always a string 7 2 Messaging in IPython 399 IPython Documentation Release 1 2 1 An Afternoon Hack Message type pyout content The counter for this execution is also provided so that clients can
559. ve Fork the ipython repository on github Making your own copy fork of ipython Then checkout and refresh master branch from main repo git checkout master git pull origin master rename pointer to main repository to upstream git remote rename origin upstream point your repo to default read write to your fork on github git remote add origin git github com your user name ipython git push up any branches you ve made and want to keep git push origin the fix im thinking of Then you can if you want follow the Development workflow 7 1 4 Git for development Contents Making your own copy fork of ipython You need to do this only once The instructions here are very similar to the instructions at http help github com forking please see that page for more detail We re repeating some of it here just to give the specifics for the ipython project and to suggest some default names 374 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack Set up and configure a github account If you don t have a github account go to the github page and make one You then need to configure your account to allow write access see the Generating SSH keys help on github help Create your own forked copy of ipython 1 Log into your github account 2 Go to the ipython github home at ipython github 3 Click on the fork button lt gt Unwatec
560. vide simple default batch templates for PBS and SGE but you may need to specify your own Here is a sample PBS script template PBS N ipython PBS j oe PBS 1 walltime 00 10 00 PBS 1 nodes n 4 ppn 4 PBS q queue cd SPBS_O_WORKDIR export PATH SHOME usr local bin export PYTHONPATH SHOME usr local lib python2 7 site packages usr local bin mpiexec n n ipengine profile dir profile_dir There are a few important points about this template 1 This template will be rendered at runtime using IPython s EvalFormatter This is simply a sub class of string Formatter that allows simple expressions on keys 2 Instead of putting in the actual number of engines use the notation n to indicate the number of engines to be started You can also use expressions like n 4 in the template to indicate the number of nodes There will always be n and profile_dir variables passed to the formatter These allow the batch system to know how many engines and where the configuration files reside The same is true for the batch queue with the template variable queue 3 Any options to ipengine can be given in the batch script template or in ipengine_config py 4 Depending on the configuration of you system you may have to set environment variables in the script template The controller template should be similar but simpler PBS N ipython PBS j oe PBS 1 walltime 00 10 00 PBS 1 node
561. wants to drop 3 1 support Incorrect parsing of raw multiline strings Test fails if from __future__ import print_function in pythonrc py nonlocal with no existing variable produces too many errors python3 is a pain minor unicode bug Yopaste in Python 3 on Mac doesn t work Error on launching Python on Win 7 and Python 2 7 3 disk IO activity on cursor press Markdown parses LaTeX math symbols as its formatting syntax in notebook display Math doesn t render tau correctly 925 Tab completion in Qt console needn t use pager 2607 2593 2602 2598 2244 2590 2581 2584 2578 2576 2560 2566 1308 1679 load_ext sympy interactive ipythonprinting dammaging output Toolbar button to open qtconsole from notebook IPython html documentation for downloading ipython notebook pylab inline replaces built in any small issue wrong printout add easier way to execute scripts in the current directory hist does not work when InteractiveShell cache_size 0 No file COPYING AttributeError module object has no attribute TestCase One of my notebooks won t load any more is there a maximum notebook size Notebook output is invisible when printing strings with rrn line endings if pyside partially present ipython qtconsole fails to load even if pyqt4 present ipython qtconsole ssh server existing hangs List command doesn t work in ipdb debugger the first time 66
562. with a single keystroke and using provides additional detail Searching through modules and namespaces with wildcards both when using the system and via the 3psearch command Completion in the local namespace by typing TAB at the prompt This works for keywords modules methods variables and files in the current directory This is supported via the readline library and full access to configuring readline s behavior is provided Custom completers can be implemented easily for different purposes system commands magic arguments etc Numbered input output prompts with command history persistent across sessions and tied to each profile full searching in this history and caching of all input and output User extensible magic commands A set of commands prefixed with is available for controlling Python itself and provides directory control namespace information and many aliases to common system shell commands Alias facility for defining your own system aliases Complete system shell access Lines starting with are passed directly to the system shell and using orvar cmd captures shell output into python variables for further use The ability to expand python variables when calling the system shell In a shell command any python variable prefixed with is expanded A double allows passing a literal to the shell for access to shell and environment variables like PATH Filesystem navigation via a
563. with map etc The controller can attempt to resume operation if it has crashed by passing ipcontroller PEsetore Engines can monitor the Hub heartbeat and shutdown if the Hub disappears for too long add HTCondor support in launchers 12 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack QtConsole Various fixes including improved performance with lots of text output and better drag and drop sup port The initial window size of the gtconsole is now configurable via IPythonWidget width and IPythonWidget height 2 3 Issues closed in the 1 0 development cycle 2 3 1 Issues closed in 1 2 GitHub stats for 2013 09 09 2014 02 21 These lists are automatically generated and may be incomplete or contain duplicates The following 13 authors contributed 84 commits Benjamin Ragan Kelley Daryl Herzmann Doug Blank Fernando Perez James Porter Juergen Hasch Julian Taylor Kyle Kelley Lawrence Fu Matthias Bussonnier Paul Ivanov Pascal Schetelat Puneeth Chaganti Takeshi Kanmae Thomas Kluyver We closed a total of 55 issues 38 pull requests and 17 regular issues this is the full list generated with the script tools github_stats py Pull Requests 38 1 2 1 e PR 4372 Don t assume that SyntaxTB is always called with a SyntaxError e PR 5166 remove mktemp usage 2 3 Issues closed in the 1 0 development cycle 13 IPython Document
564. works perfectly Currently the following color schemes are available 69 69 e NoColor uses no color escapes at all all escapes are empty safe to use in any terminal strings This scheme is thus fully e Linux works well in Linux console type environments dark background with light fonts It uses bright colors for information so it is difficult to read if you have a light colored background e LightBG the basic colors are similar to those in the Linux scheme but darker It is easy to read in terminals with light backgrounds Python uses colors for two main groups of things prompts and tracebacks which are directly printed to the terminal and the object introspection system which passes large sets of data through a pager 6 7 2 Input Output prompts and exception tracebacks You can test whether the colored prompts and tracebacks work on your system interactively by typing colors Linux at the prompt use colors LightBG if your terminal has a light background If the input prompt shows garbage like 0 32mIn 1 32m1 0 32m 0 00m instead of in color something like ID jf 3 366 Chapter 6 Configuration and customization IPython Documentation Release 1 2 1 An Afternoon Hack this means that your terminal doesn t properly handle color escape sequences You can go to a no color mode by typing colors NoColor You can try using a different terminal emu
565. x from CDN via https PR 1451 include heading level in JSON PR 1444 Fix pyhton gt python typos PR 1414 ignore errors in shell var_expand PR 1430 Fix for tornado check for tornado lt 1 1 0 PR 1413 get_home_dir expands symlinks adjust test accordingly PR 1385 updated and prettified magic doc strings PR 1406 Browser selection PR 1377 Saving non ascii history PR 1402 fix symlinked home issue for FreeBSD PR 1405 Only monkeypatch xunit when the tests are run using it PR 1395 Xunit amp KnownFailure PR 1396 Fix for tb magic PR 1386 Jsd3 PR 1388 Add simple support for running inside a virtualenv PR 1391 Improve Hub Scheduler when no engines are registered PR 1369 load header with engine id when engine dies in TaskScheduler PR 1353 Save notebook as script using unicode file handle PR 1352 Add m mod run library module as a script option PR 1363 Fix some minor color style config issues in the qtconsole 100 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack PR 1371 PR 1387 PR 1296 PR 1372 PR 1374 PR 1370 PR 1368 PR 1361 PR 1364 PR 1362 PR 1347 PR 1346 PR 1336 PR 1331 PR 1247 PR 1332 PR 1341 PR 1334 PR 1338 PR 1335 PR 1299 PR 1318 PR 1328 PR 1206 PR 1330 PR 1309 PR 1327 PR 1326 PR 1325 PR 1316 PR 1303 PR 1312 Adds a quiet ke
566. xecution on engines 0 1 2 3 In 68 y dview gather y In 69 print y 0 1 1024 59049 1048576 9765625 60466176 282475249 1073741824 5 3 IPython s Direct interface 285 IPython Documentation Release 1 2 1 An Afternoon Hack Remote imports Sometimes you will want to import packages both in your interactive session and on your remote engines This can be done with the ContextManager created by a DirectView s sync_imports method In 69 with dview sync_imports ee arees import numpy importing numpy on engine s Any imports made inside the block will also be performed on the view s engines sync_imports also takes a local boolean flag that defaults to True which specifies whether the local imports should also be performed However support for Jocal False has not been implemented so only packages that can be imported locally will work this way You can also specify imports via the require decorator This is a decorator designed for use in Depen dencies but can be used to handle remote imports as well Modules or module names passed to require will be imported before the decorated function is called If they cannot be imported the decorated function will never execute and will fail with an UnmetDependencyError Failures of single Engines will be collected and raise a CompositeError as demonstrated in the next section In 69 from IPython parallel im
567. y args tuple list The positional arguments passed to f kwargs dict The keyword arguments passed to f flags for all views block bool default view block Whether to wait for the result or return immediately False returns AsyncResult True returns actual result s of f args kwargs if multiple targets list of results matching targets track bool default view track whether to track non copying sends targets int list of ints all None default view targets Specify the destination of the job if all or None Run on all active engines if list Run on each specified engine if int Run on single engine 334 Chapter 5 Using IPython for parallel computing IPython Documentation Release 1 2 1 An Afternoon Hack Note that LoadBalanced View uses targets to restrict possible destinations LoadBalanced calls will always execute in just one location flags only in LoadBalanced Views after Dependency or collection of msg_ids Only for load balanced execution targets None Specify a list of msg_ids as a time based dependency This job will only be run after the dependencies have been met follow Dependency or collection of msg_ids Only for load balanced execution targets None Specify a list of msg_ids as a location based dependency This job will only be run on an engine where this dependency is met timeout float int or None Only for load balanced execution targets None Specify an amount of tim
568. y db is corrupted ipengines fail if clean_logs enabled Using warnings warn results in TypeError Multiprocessing in ipython notebook kernel crash ipython crashes with the following message I do not what went wrong Can you help me identify the problem 2423 2257 2418 2417 Docs typo pip install e fails rmagic can t run R s read csv on data files with NA data HTML notebook Backspace sometimes deletes multiple characters 2 3 Issues closed in the 1 0 development cycle 67 IPython Documentation Release 1 2 1 An Afternoon Hack 2275 2414 2409 2410 2366 2406 2398 2314 1688 1884 2381 1435 2372 2371 2367 2359 2227 2361 2357 2343 2315 2351 2350 2303 2330 2325 2324 1474 2318 2309 2307 2313 notebook Down_Arrow on last line of cell should move to end of line 0 13 1 does not work with current EPD 7 3 2 there is a redundant None Use usr bin python3 instead of usr bin python Notebook Dashboard notebook dir and fullpath Inability to get docstring in debugger Show line number for IndentationErrors HTML lists seem to interfere with the QtConsole display unicode exception when using run with failing script IPython embed changes color on error time doesn t work for multiline statements Add size keywords in Image class interactiveshell py misses urllib and io_open imports iPython not working Tab ex
569. y evolving as the user developer explores a problem The broad threads in computing commodity clusters multicore cloud computing etc make these capabilities of Python particularly relevant While Python is across platform tool it has particularly strong support for Windows based compute clusters running Windows HPC Server 2008 This document describes how to get started with Python on Windows HPC Server 2008 The content and emphasis here is practical installing Python configuring Python to use the Windows job scheduler and running example parallel programs interactively A more complete description of I Python s parallel computing capabilities can be found in IPython s online documentation http Apython org documentation html 5 10 2 Setting up your Windows cluster This document assumes that you already have a cluster running Windows HPC Server 2008 Here is a broad overview of what is involved with setting up such a cluster 1 Install Windows Server 2008 on the head and compute nodes in the cluster 2 Setup the network configuration on each host Each host should have a static IP address W On the head node activate the Active Directory Domain Services role and make the head node the domain controller Join the compute nodes to the newly created Active Directory AD domain Setup user accounts in the domain with shared home directories Install the HPC Pack 2008 on the head node to create a cluster
570. y thanks to Paul Ivanov for helping resolve this issue Fix IOError bug on Windows when used with gthread Work robustly if HOME is missing from environment Better POSIX support in ssh scripts remove bash specific idioms 182 Chapter 2 What s new in IPython IPython Documentation Release 1 2 1 An Afternoon Hack e Improved support for non ascii characters in log files e Work correctly in environments where GTK can be imported but not started such as a linux text console without X11 For this release we merged 24 commits contributed by the following people please let us know if we ommitted your name and we ll gladly fix this in the notes for the future e Fernando Perez e MinRK e Paul Ivanov e Pieter Cristiaan de Groot TvrtkoM 2 10 2 Release 0 10 1 Python 0 10 1 was released October 11 2010 over a year after version 0 10 This is mostly a bugfix release since after version 0 10 was released the development team s energy has been focused on the 0 11 series We have nonetheless tried to backport what fixes we could into 0 10 1 as it remains the stable series that many users have in production systems they rely on Since the 0 11 series changes many APIs in backwards incompatible ways we are willing to continue main taining the 0 10 x series We don t really have time to actively write new code for 0 10 x but we are happy to accept patches and pull requests on the Python github site If suf
571. ylab with OSX backend keyboard focus issue and hang 2107 test_octavemagic py everything fails 1212 Better understand document browser compatibility 1585 Refactor notebook templates to use Jinja2 and make each page a separate directory 1443 xticks scaling factor partially obscured with qtconsole and inline plotting 1209 can t make result work as in doc 1200 Python 0 12 Windows install fails on Vista 1127 Interactive test scripts for Qt nb issues 959 Matplotlib figures hide 2071 win32 installer issue on Windows XP 2610 ZMQInteractiveShell colors being ignored 2505 Markdown Cell incorrectly highlighting after lt 165 Installer fails to create Start Menu entries on Windows 2356 failing traceback in terminal ipython for first exception 2145 Have dashboad show when server disconect 2098 Do not crash on kernel shutdow if json file is missing 2813 Offline MathJax is broken on 0 14dev 2807 Test failure Python parallel tests test_client TestClient test_purge_everything 2486 Readline s history search in ipython console does not clear properly after cancellation with Ctrl C 2709 Cython la doesn t work 2767 What is Python utils upgradedir 2210 Placing matplotlib legend outside axis bounds causes inline display to clip it 2553 Python Notebooks not robust against client failures 2536 ImageDraw in Ipython notebook not drawing lines 2264 Feature request Versioning messaging
572. you should check is the ip address the controller is listening on and make sure that it is visible from the timing out machine See also Our notes on security in the new parallel computing code Let s say that you want to start the controller on host 0 and engines on hosts host 1 hostn The following steps are then required 1 Start the controller on host0 by running ipcontroller on host0 The controller must be instructed to listen on an interface visible to the engine machines via the ip command line argument or HubFactory ipin ipcontroller_config py 2 Move the JSON file t pcontroller engine json created by the controller from host0 to hosts host1 hostn 3 Start the engines on hosts host 1 hostn by running ipengine This command has to be told where the JSON file t pcontroller engine json is located 5 2 Starting the Python controller and engines 265 IPython Documentation Release 1 2 1 An Afternoon Hack At this point the controller and engines will be connected By default the JSON files created by the con troller are put into the IPYTHONDIR profile_default security directory If the engines share a filesystem with the controller step 2 can be skipped as the engines will automatically look at that location The final step required to actually use the running controller from a client is to move the JSON file ipcontroller client json from host0 to any host where clients will be run If these file
573. ython lib pretty py BUG Include the name of the exception type in its pretty format Fix zero copy push fix dangling buffer in Python parallel util DOC Fix references to Python lib pretty instead of the old location BUG Improve placement of CallTipWidget BUG LBYL when clearing the output history on shutdown fix sorting profiles in clustermanager BUG Fix pretty printing for overzealous objects more general fix for 662 updated magic_history docstring First version of cluster web service fix tb after SyntaxError Fix for failing testsuite when using with xml coverage on windows Add install_ext magic function Win32 shell interactivity 2 5 Issues closed in the 0 13 development cycle 99 IPython Documentation Release 1 2 1 An Afternoon Hack PR 1468 Simplify structure of a Job in the TaskScheduler PR 1447 1107 Tab autocompletion can suggest invalid syntax PR 1469 Fix typo in comment insert space PR 1463 Fix completion when importing modules in the cwd PR 1466 Fix for issue 1437 unfriendly windows qtconsole error handling PR 1432 Fix ipython directive PR 1465 allow ipython help subcommand syntax PR 1416 Conditional import of ctypes in inputhook PR 1462 expedite parallel tests PR 1410 Add javascript library and css stylesheet loading to JS class PR 1448 Fix for 875 Never build unicode error messages PR 1458 use eval to uncan References PR 1450 load mathja
574. ython object self shell ip get_ipython Declare this function as the magic mycommand ip define_magic mycommand func Type magic for more information including a list of all available magic functions at any time and their docstrings You can also type smagic_function_name see below for information on the system to get information about any particular magic function you are interested in The API documentation for the Python core magic module contains the full docstrings of all cur rently available magic commands 212 Chapter 4 Using IPython for interactive work IPython Documentation Release 1 2 1 An Afternoon Hack Access to the standard Python help Simply type help to access Python s standard help system You can also type help object for information about a given object or help keyword for information on a keyword You may need to configure your PYTHONDOCS environment variable for this feature to work correctly Dynamic object information Typing word or word prints detailed information about an object If certain strings in the object are too long e g function signatures they get snipped in the center for brevity This system gives access variable types and values docstrings function prototypes and other useful information If the information will not fit in the terminal it is displayed in a pager less if available otherwise a basic internal pager Typing word
575. yword to sync_imports Fixing Cell menu to update cell type select box Wx gui example fixes the broken example for gui wx ipcontroller cleans up connection files unless reuse True remove calls to meaningless ZMQStream on_err allow draft76 websockets Safari Ensure handler patterns are str not unicode Notebook bug fix branch avoid jsonlib returning Decimal Don t log complete contents of history replies even in debug fix weird magic completion in notebook fixups for alternate URL prefix stuff crack at making notebook html use the layout html template RST and heading cells fixes a bug causing extra newlines after comments notebook allow prefixes in URL path Don t attempt to tokenize binary files for tracebacks added key handler for control s to notebook seems to work pretty well Fix see also in docstrings so API docs build Notebook toolbar UI made notebook html extend layout html make Ctrl D in qtconsole act same as in terminal ready to merge Coverage don t preserve fixConsole output in json Add linewrapping to text cells new feature in CodeMirror Inoculate clearcmd extension into reset functionality Updatecm2 Removing Ace edit capability forgotten selected_cell gt get_selected_cell Pass subprocess test runners a suitable location for xunit output Updatecm minor heartbeat tweaks 2 5 Issues closed in the 0 13 development cycle 101 IPython Documentation Release 1 2 1 An Afternoon Hack
576. zmq IDENTITY of the upstream sockets in each Moni toredQueue lIOPub On the kernels stdout stderr are captured and published via a PUB socket These PUB sockets all connect to a SUB socket input of a MonitoredQueue which subscribes to all messages They are then republished via another PUB socket which can be subscribed by the clients Client connections The hub s registrar ROUTER socket also listens for queries from clients as to queue status and control instructions Clients connect to this socket via an DEALER during registration The Hub publishes all registration unregistration events via a PUB socket This allows clients to stay up to date with what engines are available by subscribing to the feed with a SUB socket Other processes could selectively subscribe to just registration or unregistration events 7 4 Connection Diagrams of The IPython ZMQ Cluster 413 IPython Documentation Release 1 2 1 An Afternoon Hack Client s lOPub Le eee Hub Schedulers i i i i i 1 i i i i L i i L i i L stdout err Engine s Fig 7 5 stdout err are published via a PUB SUB MonitoredQueue 414 Chapter 7 IPython developer s guide IPython Documentation Release 1 2 1 An Afternoon Hack Client s Hub Schedulers eee ee eee eee eee ee Se Le oe ee ee ee EEE EEE E E E Registration Engine s Fig 7 6 Clients connect to an ROUTER socket to query the hub 7 4 Connection

1.x PDF - Read the Docs

Contents

Download Pdf Manuals

Related Search

Related Contents