In this tutorial, we will explain IPython and its features through examples.
1. IPython Tutorial – Overview
One of Python’s most useful features is its interactive interpreter. It allows for very fast testing of ideas without the overhead of creating test files as is typical in most programming languages. However, the interpreter supplied with the standard Python distribution is somewhat limited for extended interactive use. The goal of IPython is to create a comprehensive environment for interactive and exploratory computing. To support this goal, IPython has three main components:
- An enhanced interactive Python shell.
- A decoupled two-process communication model, which allows for multiple clients to connect to a computation kernel, most notably the web-based notebook provided with Jupyter.
- An architecture for interactive parallel computing now part of the
ipyparallelpackage
2. Enhanced interactive Python shell
IPython provides an interactive shell superior to Python’s default. IPython has many features for tab completion, object introspection, system shell access, command history retrieval across sessions, and its own special command system for adding functionality when working interactively. It tries to be a very efficient environment both for Python code development and for the exploration of problems using Python objects (in situations like data analysis). It serves as an embeddable, ready-to-use interpreter for your own programs. An interactive IPython shell can be started with a single call from inside another program, providing access to the current namespace. This can be very useful both for debugging purposes and for situations where a blend of batch-processing and interactive exploration are needed.
The IPython shell offers a flexible framework that can be used as the base environment for working with other systems, with Python as the underlying bridge language. Specifically, scientific environments like Mathematica, IDL, and Matlab inspired its design, but similar ideas can be useful in many fields.
2.1 Features of interactive shell
The IPython interactive shell provides dynamic object introspection. One can access docstrings, function definition prototypes, source code, source files, and other details of any object accessible to the interpreter with a single keystroke. It allows searching through modules and namespaces with * wildcards, both when using the ? system and via the %psearch command.
It allows completion in the local namespace, by typing TAB at the prompt. This works for keywords, modules, methods, variables, and files in the current directory. This is supported via the prompt_toolkit library. Custom completers can be implemented easily for different purposes.
Numbered input/output prompts with command history (persistent across sessions and tied to each profile), full searching in this history, and caching of all input and output.
User-extensible ‘magic’ commands. A set of commands prefixed with % or %% is available for controlling IPython itself and provides directory control, namespace information, and many aliases to common system shell commands. It also provides, alias facility for defining your own system aliases.
Complete system shell access. Lines starting with ! are passed directly to the system shell and using !! or var = !cmd captures shell output into python variables for further use. The ability to expand python variables when calling the system shell. In a shell command, any python variable prefixed $ is expanded. A double $$ allows passing a literal $ to the shell (for access to shell and environment variables like PATH).
Filesystem navigation, via a magic %cd command, along with a persistent bookmark system (using %bookmark) for fast access to frequently visited directories. A lightweight persistence framework via the %store command, which allows you to save arbitrary Python variables. These get restored when you run the %store -r command. Automatic indentation and highlighting of code as you type (through the prompt_toolkit library).
Macrosystem for quickly re-executing multiple lines of previous input with a single name via the %macro command. Macros can be stored persistently %store and edited via %edit. Session logging (you can then later use these logs as code in your programs). Logs can optionally timestamp all input, and also store session output (marked as comments, so the log remains valid Python source code).
The IPython allows session restoring – logs can be replayed to restore a previous session to the state where you left it. Easier to parse visually, and in the verbose mode, they produce a lot of useful debugging information (basically a terminal version of the cgitb module).
Auto-parentheses via the %autocall command: callable objects can be executed without parentheses: sin 3 is automatically converted to sin(3). Using , or ; as the first character forces auto-quoting of the rest of the line: ,my_function a b becomes automatically my_function("a","b"), while ;my_function a b becomes my_function("a b")
You can define filters that pre-process user input to simplify input in special situations. This allows for example pasting multi-line code fragments which start with >>> or … such as those from other python sessions or the standard Python documentation.
A Flexible configuration system uses a configuration file that allows permanent setting of all command-line options, module loading, code, and file execution. The system allows recursive file inclusion, so you can have a base file with defaults and layers which load other customizations for particular projects.
You can call IPython a python shell inside your own python programs. This can be used both for debugging code or for providing interactive abilities to your programs with knowledge about the local namespaces.
You can set IPython to call up an enhanced version of the Python debugger (pdb) every time there is an uncaught exception. This drops you inside the code which triggered the exception with all the data live and it is possible to navigate the stack to rapidly isolate the source of a bug. The %run magic command (with the -d option) can run any script under pdb’s control, automatically setting initial breakpoints for you. This version of pdb has IPython-specific improvements, including tab-completion and traceback coloring support. For even easier debugger access, try %debug after seeing an exception.
Profiler support. You can run single statements (similar to profile.run()) or complete programs under the profiler’s control. While this is possible with standard cProfile or profile modules, IPython wraps this functionality with magic commands (see %prun and %run -p) convenient for rapid interactive work.
Simple timing information. You can use the %timeit command to get the execution time of a Python statement or expression. This machinery is intelligent enough to do more repetitions for commands that finish very quickly in order to get a better estimate of their running time. To get the timing information for more than one expression, use the %%timeit cell magic command.
3. Decoupled two-process model
IPython has abstracted and extended the notion of a traditional Read-Evaluate-Print Loop (REPL) environment by decoupling the evaluation into its own process. It is called a kernel: it receives execution instructions from clients and communicates the results back to them. This decoupling allows to have several clients connected to the same kernel, and even allows clients and kernels to live on different machines. With the exclusion of the traditional single process terminal-based IPython (what you start if you run ipython without any subcommands), all other IPython machinery uses this two-process model. Most of this is now part of the Jupyter project. This means that when you start jupyter qtconsole, you’re really starting two processes, a kernel and a Qt-based client which can send commands to and receive results from that kernel.
If there is already a kernel running that you want to connect to, you can pass the --existing flag which will skip initiating a new kernel and connect to the most recent kernel, instead. To connect to a specific kernel once you have several kernels running, use the %connect_info magic to get the unique connection file, which will be something like --existing kernel-19732.json but with different numbers which correspond to the Process ID of the kernel.
4. Installation
This section will guide you through the installation process of IPython. I am using Mac so the process will be specific to it, for other operating systems the process might be a bit different. IPython and most dependencies should be installed via pip. In many scenarios, this is the simplest method of installing Python packages. IPython relies on a number of other Python packages. Installing using a package manager like pip or conda will ensure the necessary packages are installed. Manual installation without dependencies is possible, but not recommended. The dependencies can be viewed with package manager commands, such as pip show ipython
~$ pip3 show ipython
Name: ipython
Version: 7.30.1
Summary: IPython: Productive Interactive Computing
Home-page: https://ipython.org
Author: The IPython Development Team
Author-email: ipython-dev@python.org
License: BSD
Location: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages
Requires: backcall, appnope, matplotlib-inline, jedi, pygments, setuptools, prompt-toolkit, decorator, pickleshare, pexpect, traitlets
Required-by: jupyter-console, ipywidgets, ipykernel
Run the below command to install IPython:
~$ pip3 install ipython
This installs IPython as well as its dependencies. If you want to use IPython with notebooks or the Qt console, you should also install Jupyter
~$ pip3 install jupyter
Install and register an IPython kernel with Jupyter:
~$ python3 -m pip install ipykernel
~$ python3 -m ipykernel install
To check if IPython is installed correctly type ipython
~$ ipython
Python 3.8.1 (v3.8.1:1b293b6006, Dec 18 2019, 14:08:53)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.30.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]:
5. Example
Unlike the Python REPL, you will see that the input prompt is In [N]: instead of >>>. You should be able to type single-line expressions and then press enter to evaluate them. If an expression is incomplete, IPython will automatically detect this and add a new line when you press Enter instead of executing right away. Unlike many other REPLs, with IPython you can use the up and down arrow keys when editing multi-line code blocks.
In [1]: 2*3*4
Out[1]: 24
In [2]: def helloWorld():
...: print('Hello World!')
...:
In [3]: helloWorld()
Hello World!
In [4]:
You will see the difference with the standard Python REPL. The syntax in your code will be highlighted, you will also see that the output line starts with Out. Depending on the exact command you are typing you might realize that sometimes Enter will add a new line, and sometimes it will execute the current statement. IPython tries to guess what you are doing, so most of the time you should not have to care. Though if by any chance IPython does not the right thing you can force the execution of the current code block by pressing in sequence Esc and Enter. You can also force the insertion of a new line at the position of the cursor by using Ctrl-o.
Below are some of the most helpful commands you will need when learning to use IPython:
| Command | Description |
| ? | Introduction and overview of IPython’s features. |
| %quickref | Quick reference. |
| help | Python’s own help system. |
| object? | Details about ‘object’, use ‘object??’ for extra details. |
6. Summary
In this tutorial, we learned about IPython. We looked at how easy it is to install it. We also discussed some of its features then looked at two of the most important and common features – interactive shell and two-process model. In the end, we looked at some of the most helpful commands and some working examples.
IPython stores both the commands you enter, and the results it produces. You can easily go through previous commands with the up and down arrow keys, or access your history in more sophisticated ways.
