Reusing code from different IPython notebooks

Question

I am using IPython and want to run functions from one notebook from another (without cutting and pasting them between different notebooks). Is this possible and reasonably e

Answer 1

Here are two additional tips:

1. You can also run %qtconsole magic directly from the notebook and it will automatically connect to the notebook kernel.

2. Check out https://github.com/atiasnir/ipnb

   You can use it to import notebook files as if they're standard python modules (I'm the author :-)). Main limitation here is that it will discard magic cells (because it does not use IPython at all) but otherwise it should work fine.

Answer 2

Yes, you can "run functions from one notebook from another (without cutting and pasting them between different notebooks)" -- and, yes, it's easy to do!

tl;dr: put the code in python files (*.py) in the file system & let multiple notebooks use the same code. (It's that simple.)

(Why put so much code in notebooks, when we have perfectly good code editors & IDEs that are so much better for writing & reading code? Not to mention the need for proper version control! What are we trying to achieve, and at what expense? </rant>)

Details:

• Put your code in normal python files, eg my_code/foo.py, adding a (probably empty) my_code/__init__.py
• Take advantage of having the code under proper version control (eg git) -- notice how hard it was to diff ipynb json files?
• Also put the notebooks also under version control. Raw git logs will be hard to read, but the comments can be useful. (GitHub/GitLab displays ipython notebooks, btw.)
• Limit the py source in the ipynb notebook to small amounts of "driver" code, and output, and documentation.
• See also: https://ipython.org/ipython-doc/stable/config/extensions/autoreload.html
• If you want to "inline" the external python files, just use (for example) magic %cat my_code/foo.py ...

...If you want something fancier to display that source inline (optionally, adding the following to an external, reusable source file)...

import IPython
from pygments import highlight
from pygments.formatters import HtmlFormatter
from pygments.lexers import get_lexer_for_filename

filename='my_code/foo.py'
with open(filename) as f: code = f.read()
formatter = HtmlFormatter(linenos='inline')
IPython.display.HTML('<style type="text/css">{}</style>{}'.format(
                formatter.get_style_defs('.highlight'),
                highlight(code, get_lexer_for_filename(filename), formatter)))

Your favorite code editors & IDEs thank you for your support.

Answer 3

Ipythons %run magic allows you execute python files and ipython scripts in a notebook. I sometimes use the -i option so it runs in the notebooks namespace. Execute a cell with %run? in it for more info.

You can use the ipython --script to save notebooks also as .py files on each save or uncomment the line c.NotebookManager.save_script=True in your ipython_notebook_config.py file for the same effect (use ipython profile create for setting that up - on Ubuntu the config files live in ~/.config/ipython/).

Edit: The following is true, but unnecessary - you can %run a .ipynb file directly. Thanks Eric.

If you use ipython magics in the notebook you want to import, I found that you can rename the .py file to .ipy (an ipython script), but I had to remove the first line (which contained the file encoding declaration) for it to work. There is probably a better way! This approach will likely confuse cell magics too (they'd all get applied at once).

Answer 4

There is also a "write and execute" extension, which will let you write the content of a cell to a file (and replace old content -> update code), which can then be imported in another notebook.

https://github.com/minrk/ipython_extensions#write-and-execute

In one notebook (two cells)

%reload_ext writeandexecute
--
%%writeandexecute -i some_unique_string functions.py
def do_something(txt):
    print(txt)

And then in the other notebook:

from functions import do_something
do_something("hello world")

Answer 5

I do call notebooks from other notebooks. You can even pass "parameters" to other notebooks using the following trick:

Place params dictionary in the first cell of "report_template.ipynb".

params = dict(platform='iOS', 
              start_date='2016-05-01', 
              retention=7)
df = get_data(params ..)
do_analysis(params ..)

And in another (higher logical level) notebook, execute it using this function:

def run_notebook(nbfile, **kwargs):
    """
    example:
    run_notebook('report.ipynb', platform='google_play', start_date='2016-06-10')
    """

    def read_notebook(nbfile):
        if not nbfile.endswith('.ipynb'):
            nbfile += '.ipynb'

        with io.open(nbfile) as f:
            nb = nbformat.read(f, as_version=4)
        return nb

    ip = get_ipython()
    gl = ip.ns_table['user_global']
    gl['params'] = None
    arguments_in_original_state = True

    for cell in read_notebook(nbfile).cells:
        if cell.cell_type != 'code':
            continue
        ip.run_cell(cell.source)

        if arguments_in_original_state and type(gl['params']) == dict:
            gl['params'].update(kwargs)
            arguments_in_original_state = False

run_notebook("report_template.ipynb", start_date='2016-09-01')

This command will execute each cell of the "report_template" notebook and will override relevant keys of params dictionary starting from the second cell

Answer 6

I use the following function in the notebook from which I want to load functions or actions from a source notebook:

import io
import nbformat

def execute_notebook(nbfile):
    with io.open(nbfile, encoding="utf8") as f:
        nb = nbformat.read(f, as_version=4)

    ip = get_ipython()

    for cell in nb.cells:
        if cell.cell_type != 'code':
            continue
        ip.run_cell(cell.source)

Use like:

execute_notebook(path/to/notebook.ipynb)