These Methods Will Change How You Use Pandas

Make Pandas Faster with a Simple Change

Written by: Amal Hasni & Dhia Hmila

Pandas is the go-to Python package for manipulating data. It provides multiple methods to perform different operations on DataFrame objects. In this article, we'll discuss different ways of efficiently filtering data and creating new columns.

First, let’s create a dataset:

It should look something like this, where each line describes the height, weight, and hip circumference of an individual:

Suppose, we want to only keep overweight individuals based on the BMI (Body Mass Index) to perform further analysis. We want to filter our Dataframe and then create a new copy.

This is fairly easy to do in classic pandas:

The alternative way of doing this is to use the query method:

You notice that instead of using regular python to define our filter, we use an expression that is directly evaluated by pandas (or to be precise numexpr which is the engine used by pandas).

Why would you want to use this?

Related posts:

I am a 55 year old person of so called “indian” descent, born and bred in South Africa. Apartheid was abolished when I was 28, so I had a good long period growing up under it. My great grandfather…

I started my podcast in February 2020. At the time, I was working full time for an e-commerce company and teaching Japanese part time, but I decided to quit my 9–5 office job and teach Japanese full…

Modified atmospheric packaging (MAP) is an effective technology extending the shelf life of packaged foods without the need for added preservatives. Therefore, this type of packaging is used for…