Python language is one of the best coding languages that you can start handling for your first data science project. This is a fantastic language that capable to take on all of the work that you want to do with data science and has the power that is needed to help create some great machine learning algorithms. With that said, it is still a great option for beginners because it has been designed to work with those who have never done programming before. While you can choose to work with the R programming language as well, you will find that the Python language is one of the best options because of its ease of use and power that combines.
Programming languages help us to expand our theoretical knowledge to something that can happen. Data science, which usually needs a lot of data to make things happen, will by nature take advantage of programming languages to make the data organize well for further steps of the model development. So, let us start learning about Python for a better understanding of the topic.
Why Python Is Important?
To illustrate this problem more vividly, we might as well assume that we have a small partner named Estella. She just got a job related to Data Science after graduating from the math department. On her first day at work, she was enthusiastic and eager to get in touch with this dude-new industry. But she soon found herself facing a huge difficulty.
The data needed to process the work is not stored in her personal computer, but in remote servers, some in traditional relational databases, and some in Hadoop clusters. Unlike Windows, which is mostly used by personal computers, Linux-like systems are used on remote servers. Estella is not used to this operating system because the familiar graphical interface is missing. All operations, such as the simplest reading of files, need to be programmed by oneself. Therefore, Estella is eager to find a programming language that is simple to write, easy to learn, and easy to use.
What is more fatal is that the familiar data modeling software, such as SPSS and MATLAB, cannot be used in the new working environment. However, Estella often uses some basic algorithms provided by this software in her daily work, such as linear regression and logical regression. Therefore, she hopes that the programming language she finds will also have a library of algorithms that can be used easily, and of course, it is better to be free of charge.
The whole process is very similar to Estella’s favorite table tennis. The assumption is sent to the data as a “ball”, and then the adjustment is made according to the “return ball” of the data, and the above actions are repeated. Therefore, Estella added one more item to her request: the programming language can be modified and used at any time without compilation. It is better to have an immediate response command window so that she can quickly verify her ideas. After a search, Estella excitedly told everyone that she had found an IT tool that met all her requirements that is Python. I hope you have got a good layman introduction on why programming language is important for Data Science.
What Is Python?
Python is an object-oriented and interpretive computer program language. Its syntax is simple and contains a set of standard libraries with complete functions, which can easily accomplish many common tasks. Speaking of Python, its birth is also quite interesting. During the Christmas holidays in 1989, Dutch programmer Guido van Rossum stayed at home and found himself doing nothing. So, to pass the “boring” time, he wrote the first version of Python.
Python is widely used. According to statistics from GitHub, an open-source community, it has been one of the most popular programming languages in the past 10 years and is more popular than traditional C, C + + languages, and C# which is very commonly used in Windows systems. After using Python for some time, Estella thinks it is a programming language specially designed for non-professional programmers.
Its grammatical structure is very concise, encouraging everyone to write as much code as possible that is easy to understand and write as little code as possible.
Functionally speaking, Python has a large number of standard libraries and third-party libraries. Estella develops her application based on these existing programs, which can get twice the result with half the effort and speed up the development progress.
Python’s Position in Data Science
After mastering Python as a programming language, Estella can do many interesting things, such as writing a web crawler, collecting needed data from the Internet, developing a task scheduling system, updating the model regularly, etc.
Below we will describe how the Python is used by Estella for Data Science applications:
- Data Cleaning
After obtaining the original data, Estella will first do preliminary processing on the data, such as unifying the case of the string, correcting the wrong data, etc. This is also the so-called “clean up” of “dirty” data to make the data more suitable for analysis. With Python and its third-party library pandas, Estella can easily complete this step of work. - Data Visualization
Estella uses Matplotlib to display data graphically. Before extracting the features, Estella can get the first intuitive feeling of the data from the graph and enlighten the thinking. When communicating with colleagues in other departments, information can be clearly and effectively conveyed and communicated with the help of graphics so that those insights can be put on paper. - Feature Extraction
In this step, Richard usually associates relevant data stored in different places, for example, integrating customer basic information and customer shopping information through customer ID. Then transform the data and extract the variables useful for modeling. These variables are called features. In this process, Estella will use Python’s NumPy, SciPy, pandas, and PySpark. - Model Building
The open-source libraries sci-kit-learn, StatsModels, Spark ML, and TensorFlow cover almost all the commonly used basic algorithms. Based on these algorithm bases and according to the data characteristics and algorithm assumptions, Estella can easily build the basic algorithms together and create the model she wants.
The above four things are also the four core steps in Data Science. No wonder Estella, like most other data scientists, chose Python as a tool to complete his work.
Python Installation
After introducing so many advantages of Python, let’s quickly install it and feel it for ourselves.
Python has two major versions: Python 2 and Python 3. Python 3 is a higher version with new features that Python 2 does not have. However, because Python 3 was not designed with backward compatibility in mind, Python 2 was still the main product in actual production. Therefore, it is recommended that readers still use Python 2 when installing completely.
It should be noted that the distributed Machine Learning library Spark ML involves the installation of Java and Scala, and will not be introduced here for the time being.
Installation Under Windows
The author does not recommend people to develop under Windows system. There are many reasons, the most important of which is that in the era of big data, as mentioned by Estella earlier, data is stored under the Linux system. Therefore, in production, the programs developed by data scientists will eventually run in the Linux environment. However, the compatibility between Windows and Linux is not good, which easily leads to the development and debugging of good programs under Windows, and cannot operate normally under the actual production environment.
If the computer the reader uses is a Windows system, he can choose to install a Linux virtual machine and then develop it on the virtual machine. If readers insist on using Windows, due to the limitation of TensorFlow under Windows, they can only choose to install Python 3. Anaconda installed several applications under Windows, such as IPython, Jupyter, Conda, and Spyder. Let’s explore some of them in detail:
Conda
It is a management system for the Python development environment and open source libraries. If readers are familiar with Linux, Conda is equivalent to pip + virtualenv under Linux. Readers can list installed Python libraries by entering “Condolist” on the command line.
Spyder
It is an integrated development environment (IDE) specially designed for Python for scientific computing. If readers are familiar with the mathematical analysis software MATLAB, they can find that Spyder and MATLAB are very similar in syntax and interface.
Installation Under MAC
Like Anaconda’s version of Windows, Anaconda’s Mac version does not contain a deep learning library TensorFlow, which needs to be installed using pip (Python Package Management System). Although using pip requires a command line, it is very simple to operate and even easier than installing Anaconda. Moreover, pip is more widely used, so it is suggested that readers try to install the required libraries with pip from the beginning. The installation method without Anaconda is described below.
Starting with Mac OS X 10.2, Python is preinstalled on macs. For learning purposes, you can choose to use the pre-installed version of Pythonï¼› directly. If it is for development purposes, pre-installed Python is easy to encounter problems when installing third-party libraries, and the latest version of Python needs to be reinstalled. The reader is recommended to reinstall Python here.
Installation Under Linux
Similar to Mac, Anaconda also offers Linux versions. Please refer to the instructions under Windows and the accompanying code for specific installation steps.
There are many versions of Linux, but due to space limitations, the only installation on Ubuntu is described here. The following installation guide may also run on other versions of Linux, but we have only tested these installation steps on Ubuntu 14.04 or later. Although Ubuntu has pre-installed Python, the version is older, and it is recommended to install a newer version of Python.
Install Python
install [insert command here]
Pip is a Python software package management system that facilitates us to install the required third-party libraries. The steps for installing pip are as follows:
- Open the terminal
- Enter and run the following code
Python shell
Python, as a dynamic language, is usually used in two ways: it can be used as a script interpreter to run edited program scripts; At the same time, Python provides a real-time interactive command window (Python shell) in which any Python statement can be entered and run. This makes it easy to learn, debug, and test Python statements.
Enter “Python” in the terminal (Linux or Mac) or command prompt (Windows) to start the Python shell.
- You can assign values to variables in the Python shell and then calculate the variables used. And you can always use these variables as long as you don’t close the shell. As shown in lines 1 to 3 of the code. It is worth noting that Python is a so-called dynamic type language, so there is no need to declare the type of a variable when assigning values to variables.
- Any Python statement can be run in the Python shell, as shown in the code, so some people even use it as a calculator.
- You can also import and use a third-party library in the shell, as shown. It should be noted that as shown in the code, the third-party library “NumPy” can be given an alias, such as “np” while being imported. When “NumPy” is needed later, it is replaced by “np” to reduce the amount of character input.
