Inroduction to Python and Data Science

I was exploring Linkedin profiles of well known data scientists like Jeff Hammerbacher,  Hilary Mason, DJ Patil, Gilad Lotan to get the idea about their technical skill-set. The first common thing I could find was Python. So I decided to explore about general projects and data mining/ machine learning libraries associated with Python.

General Python Projects:

Python: A general purpose high-level programming language. Python supports multiple programming paradigms, like object-oriented, imperative and functional programming or procedural styles. [1] Python implementation is under open source license that makes it freely usable and distributable, even for commercial use. [2]

Created by: Guido van Rossum

Maintained by: Python Software Foundation

CPython: It is the default, most widely used implementation of Python.

Written in: C

Maintained by: Python core developers and the Python community, supported by the Python Software Foundation

Difference between Python and CPython: Python is programming language and CPython is default implementation of it. So when we generally refer python programming language we are talking about CPython. There are several other implementations as well like Jython, IronPython etc.

Jython: Implementation of Python in Java. It has several differences and incompatibilities with CPython.

Written in: Java and Python

Successor of : JPython

RPython and PyPy: RPython (restricted python), is restricted subset of python. PyPy is interpreter which is written in RPython.

Project goal: Speed, efficiency and compatibility of CPython interpreter.

IronPython: It is Python implementation targeted at .NET framework. 

Written in: C#

Created by: Jim Hugunin

Currently maintained by: Volunteers at Microsoft's CodePlex open-source repository

Cython: It enables to write Python code which can be called back and forth, from and to C or C++ code natively. It is nothing but C extension for python.

IPython: It is interactive python. Motivation is scientific imputing and exploratory analysis, where we can directly play with data/ files. Default interactive environment is having limited functionality issue which can be solved by IPython.

Created by: Fernando Perez and others.

Getting started: IPython: Python at your fingertips (talk at Pycon 2012 by IPython creators)

Specific project related to Data (Processing, Mining, Visualization and Machine Learning):

SciPy Ecosystem: It is computing environment and open source ecosystem of Python packages used by scientists, analysts and engineers for performing scientific and technical computing.

• Pandas: Python library providing high-performance, easy-to-use data structures and data analysis tools. 
• NumPy: NumPy is python library that supports large, multi-dimensional arrays and high-level mathematical functions to perform various operations on these arrays. [written in: C and Python]
• SciPy: It also refers to a python package (library) of algorithms and mathematical functions which is a core element of the SciPy environment for technical computing.
• matplotlib: A python library which is used for 2D plotting (used for creation of various types graphs and charts)
• IPython: The IPython project mentioned above is also part of core SciPy stack.
• scikit: It is another package of python for scientific computing. This is not a core part of SciPy but add-on package.

StatsModels: It is a python module that enables users to explore data, estimate statistical models and perform statistical tests.

scikit-learn: Open source machine learning library build on top of NumPy, SciPy and matplotlib. Note that it is different from Scikit.

PypeR: It enables us to use R (most preferred language of data scientists) in Python through PIPE.

NetworkX: Python package for the creation, manipulation and study of the structure and functions of complex networks.

Note: I wish to make this post comprehensive over the time, so please post comments if I am missing any great projects here.