Edit Question - KnowledgeHub

Title Body

I have a scenario where a user wants to apply several filters to a Pandas DataFrame or Series object . Essentially, I want to efficiently chain a bunch of filtering (comparison operations) together that are specified at run-time by the user. The filters should be additive (aka each one applied should narrow results). I'm currently using reindex() (as below) but this creates a new object each time and copies the underlying data (if I understand the documentation correctly). I want to avoid this unnecessary copying as it will be really inefficient when filtering a big Series or DataFrame. I'm thinking that using apply() , map() , or something similar might be better. I'm pretty new to Pandas though so still trying to wrap my head around everything. Also, I would like to expand this so that the dictionary passed in can include the columns to operate on and filter an entire DataFrame based on the input dictionary. However, I'm assuming whatever works for a Series can be easily expanded to a DataFrame. TL;DR I want to take a dictionary of the following form and apply each operation to a given Series object and return a 'filtered' Series object. relops = {'>=': [1], '<=': [1]} Long Example I'll start with an example of what I have currently and just filtering a single Series object. Below is the function I'm currently using: def apply_relops(series, relops): """ Pass dictionary of relational operators to perform on given series object """ for op, vals in relops.iteritems(): op_func = ops[op] for val in vals: filtered = op_func(series, val) series = series.reindex(series[filtered]) return series The user provides a

Tags (comma-separated) Cancel