How does Great Expectations fit into your data stack?

The Great Expectations community has proven to be a resourceful one that we’ve learned so much from.

We would love to know where and how you integrate Great Expectations into your data pipeline?

Where were the points of friction in the integration?

What would you like to see improved about the process?

My plan is to integrate GE in a Cloudera Hadoop System. Right now I’m figuring out what the best way would be to integrate things with Oozie (I’d prefer Airflow, but customer works with Oozie).
In Germany, bigger mid-size companies are still a lot on premises.
In the end I’d like to use DataDocs, too so have to figure out where to deploy all that results and how to do alerting (don’t know yet if letting do GE or Oozie the E-Mail after failure is better).

1 Like