Let’s say I do this and after 1 year or so of using great expectations on daily basis if I run the following command
great_expectations --v3-api docs build --site-name gs_site
Will great expectations download everything from gcs to re-build the data docs? I am worried that re-building data docs over time will cause our cost on GCS to increase. Because great expectations will keep on hitting GCS to get all validations from past one year.
I considered setting up postgres for the validation result storage as per this page but this does not tell me what table structure do I need to create in this schema. Will great expectations create the table structure for me when I run this for the first time? Or do I need to run some sql for setting it up? If yes, where can I find the sql to setup postgres.
I would prefer GCS but I don’t want to get into having an ever-increasing cost due to re-building the index page for the data docs.