I was introduced to Great Expectations just yesterday so this is going to be a beginner-level, high-level question.
Right now we are focusing on cleaning our Pandas DataFrame datasets (just cleaning, no transform) by defining functions that should match our criteria. It looks like we can accomplish this by using the “expectations” because they return
unexpected_index_list, which we can use it to filter out unwanted data. We are also planning to implement custom “expectations” within Great Expectations.
Are we gaining any benefit from cleaning our data this way? Or should we just do this cleaning process outside Great Expectations with our own pure Python functions? I am asking to plan our future workflows.