Bring New Life to Legacy Code with Anaconda Modernize Legacy Code with the Anaconda Platform T.J. Alumbaugh Continuum Analytics Webinar June 8, 2016 © 2016 Continuum Analytics
Bring New Life to Legacy Code with AnacondaModernize Legacy Code with the Anaconda Platform
T.J. Alumbaugh
Continuum Analytics WebinarJune 8, 2016
© 2016 Continuum Analytics
2
Today’s Presenter: T.J. Alumbaugh
© 2016 Continuum Analytics
T.J. AlumbaughSenior Solutions Architect
Continuum Analytics
T.J. Alumbaugh is a Senior Solutions Architect at Continuum Analytics, where he helps many of the world's largest companies solve their most challenging computational problems.
Before Continuum, T.J. worked as a software engineer, researcher and computer scientist, most notably at Lawrence Livermore National Laboratory and Sandia National Laboratories.
He holds a Masters degree in Computer Science from the University of Illinois, as well as Bachelors degrees in Mathematics and Computer Science.
3
Objectives Today• Scenario 1: Tested and Validated Legacy Analytics Need to Deployed to Windos & OS X
– Make legacy code available to your entire team– Create conda packages from Fortran code and distribute across multiple platforms
• Scenario 2: Create a “Point and Click” Interface to Legacy Code– Use your legacy code in Anaconda to create modern, interactive visualization– Create and launch a Bokeh visualization that calls Fortran code
• Scenario 3: Create performant image filters in Python & Pass to Legacy C Application– Extend the life of C code by passing python functions as a callback– Pass Numba-compiled functions as a callback to C code
© 2016 Continuum Analytics
DEPLOY LEGACY CODE TO WINDOWS & OS X
Scenario 1
© 2016 Continuum Analytics
5
Scenario 1: Deploy a Legacy Fortran Application with Anaconda
• Legacy codebases in Fortran and C/C++ exist in many large organizations
• Sometimes several hundred KLOC representing decades-worth of person hours
• Regulatory issues, cost and inertia often mean that this code is not going anywhere soon!
© 2016 Continuum Analytics
6
Scenario 1: Deploy a Legacy Fortran Application with Anaconda
• What if developers and analysts not trained in Fortran/C/C++ could take advantage of these applications?
• What if these applications could be compiled once, and then reproducibly deployed across multiple platforms to any machine in your organization?
© 2016 Continuum Analytics
7
Scenario 1: Deploy a Legacy Fortran Application with Anaconda
Demo: Package a Legacy Fortran application on Windows
1. Annotate the source code so that Python understands the inputs/outputs of the function
2. Build a conda package using a conda recipe that will work on multiple platforms
3. Upload the packages to a package server
© 2016 Continuum Analytics
8
Scenario 1: Deploy a Legacy Fortran Application with Anaconda
Tools we will use:
1. Compilers (MS Visual Studio 2008 and Intel Fortran compiler)
2. F2py: tool included with NumPy that helps map Fortran functions to NumPy/Python equivalent
3. Conda-build
4. Anaconda-client
© 2016 Continuum Analytics
9
Scenario 1: Deploy a Legacy Fortran Application with Anaconda
© 2016 Continuum Analytics
10
Scenario 1: Deploy a Legacy Fortran Application with Anaconda
Additional Notes:
• Same process for C/C++ application, different wrapping technolgoy (SWIG, Cython)
• Conda build steps and conda recipe remain the same
© 2016 Continuum Analytics
CREATE A “POINT AND CLICK” INTERFACE TO LEGACY CODE
Scenario 2
© 2016 Continuum Analytics
12
Scenario 2: Embed Legacy Code in an Interactive Visualization with Bokeh
• Modern business tools are increasingly web-based or SaaS in nature.
• Some groups focused on your core business (and not tech) could benefit from having access to these legacy applications.
© 2016 Continuum Analytics
13
Scenario 2: Embed Legacy Code in an Interactive Visualization with Bokeh
• What if a business analyst could have a “point and click” interface to legacy compiled code?
• What if you could easily deploy a web application within your organization to provide such an interface through a standard browser?
© 2016 Continuum Analytics
14
Scenario 2: Embed Legacy Code in an Interactive Visualization with Bokeh
Demo: Run Fortran code inside a web application1. Install bokeh in your conda environment2. Create script with UI elements and one
callback function to call Fortran code from conda package
3. Serve bokeh application with bokeh
© 2016 Continuum Analytics
15
Scenario 2: Embed Legacy Code in an Interactive Visualization with Bokeh
Tools we will use:
1. conda
2. Bokeh
© 2016 Continuum Analytics
16
Scenario 2: Embed Legacy Code in an Interactive Visualization with Bokeh
Demo:
© 2016 Continuum Analytics
17
Scenario 2: Embed Legacy Code in an Interactive Visualization with Bokeh
And Bokeh works on mobile devices as well!
© 2016 Continuum Analytics
18
Scenario 2: Embed Legacy Code in an Interactive Visualization with BokehAdditional Notes:
- Some amount of network configuration is typically needed to allow others to access the web application running locally on your machine.
- `bokeh serve` is great for a quick way to deploy a “POC” web application. For information on a “real” web deployment with bokeh, see the bokeh docs.
© 2016 Continuum Analytics
PASS PYTHON CODE AS A C CALLBACK
Scenario 3
© 2016 Continuum Analytics
20
Scenario 3: Pass Python Code as a C Callback
• Often, C/C++ libraries have APIs that take “callbacks” (functions that are called internal to the library)
• The legacy libraries have great utility, but it doesn’t make sense to continue to develop in the legacy language
• Writing “safe” code in C/C++ requires expertise that is often hard to find
© 2016 Continuum Analytics- Confidential & Proprietary
21
Scenario 3: Pass Python Code as a C Callback • What if you could leverage existing
NumPy/Python expertise in your organization to create code assets in Python?
• What if you could stop writing more code in the old language, but not sacrifice speed?
© 2016 Continuum Analytics
22
Scenario 3: Pass Python Code as a C Callback
Demo: Create a Numba-compiled C-callback and pass it to a function in an existing C image processing library1. Compile C code as a shared library2. Load shared library into Python3. Compile a Python function with a Numba decorator4. Call C function and pass the Numba function as an
argument to the C function
© 2016 Continuum Analytics
23
Scenario 3: Pass Python Code as a C Callback
Tools we will use:
1. Compiler: gcc on Mac OS X
2. numba
3. ctypes (comes with Python)
© 2016 Continuum Analytics
24
Scenario 3: Pass Python Code as a C Callback
Gaussian Filter in C and in Python - Common image processing technique - Replace pixel value with weighted sum of neighboring pixels - Reduces noise in an image
© 2016 Continuum Analytics
25
Scenario 3: Pass Python Code as a C Callback
Demo: Implement Gaussian Filter in Python, Call in C
© 2016 Continuum Analytics
26
Scenario 3: Pass Python Code as a C Callback
Additional Notes:
- Specify argument types and result type for each function you call through the “ctypes” technique.
- Set the `errcheck` property on a foreign function. This can be used to raise an exception on return of the foreign function if the result is not as expected. See the ctypes documentation for more details.
© 2016 Continuum Analytics
SUMMARYWhat We Learned Today
© 2016 Continuum Analytics- Confidential & Proprietary
28
Summary: What We Learned Today• Created conda packages from Fortran code and uploaded to
Anaconda Cloud
• Created and launched an interactive Bokeh visualization that calls Fortran signal processing code
• Passed a Numba-compiled function as a callback to a C library to do Gaussian filtering in Python
© 2016 Continuum Analytics
LEARN MORE & GO FURTHEREngage with Continuum
© 2016 Continuum Analytics- Confidential & Proprietary
30
Share conda Packages Behind the Firewall
© 2016 Continuum Analytics
You can share and manage conda packages behind your firewall with the Repository server in Anaconda Enterprise and Anaconda Workgroup.
Contact us at [email protected] for more information, or visit https://www.continuum.io/anaconda-subscriptions.
31
Learn More About conda & Numba
You can learn more about conda, Numba and Continuum Analytics’ other open source projects here:• http://conda.pydata.org/docs/• http://numba.pydata.org/
© 2016 Continuum Analytics
32
Engage with Continuum’s Team
You can hire Continuum’s team of solution architects, consultants, and Python experts to integrate your legacy applications into a modern analytics platform.
Contact us at [email protected] for more information
© 2016 Continuum Analytics
THANK YOU!Please email [email protected] for more information
© 2016 Continuum Analytics- Confidential & Proprietary 33