review comment changes

guptadivyank · guptadivyank · commit 9e2ab5bd06c2 · 2023-07-11T01:21:31.000+05:30
diff --git a/ads/feature_store/docs/source/dataset.rst b/ads/feature_store/docs/source/dataset.rst
@@ -124,6 +124,71 @@ With a Dataset instance, we can get the last dataset job details using ``get_las
   df = dataset_job.get_validation_output().to_dataframe()
   df.show()
 
+Save expectation entity
+=======================
+Feature store allows you to define expectations on data being materialized into feature group instance. With a ``FeatureGroup`` instance, we can save the expectation entity using ``save_expectation()``
+
+
+.. image:: figures/validation.png
+
+The ``.save_expectation()`` method takes the following optional parameter:
+
+- ``expectation: Expectation``. Expectation of great expectation
+- ``expectation_type: ExpectationType``. Type of expectation
+        - ``ExpectationType.STRICT``: Fail the job if expectation not met
+        - ``ExpectationType.LENIENT``: Pass the job even if expectation not met
+
+.. code-block:: python3
+
+  feature_group.save_expectation(expectation_suite, expectation_type="STRICT")
+
+For more details on expectation please refer :ref:`Feature Validation`
+
+Statistics Computation
+========================
+During the materialization feature store performs computation of statistical metrics for all the features  by default. This can be configured using ``StatisticsConfig`` object which can be passed at the creation of
+dataset or it can be updated later as well.
+
+.. code-block:: python3
+
+  # Define statistics configuration for selected features
+  stats_config = StatisticsConfig().with_is_enabled(True).with_columns(["column1", "column2"])
+
+
+This can be used with dataset instance.
+
+.. code-block:: python3
+
+  from ads.feature_store.dataset import Dataset
+
+  dataset = (
+        Dataset
+        .with_name("<dataset_name>")
+        .with_entity_id(<entity_id>)
+        .with_feature_store_id("<feature_store_id>")
+        .with_description("<dataset_description>")
+        .with_compartment_id("<compartment_id>")
+        .with_dataset_ingestion_mode(DatasetIngestionMode.SQL)
+        .with_query('SELECT col FROM <entity_id>.<feature_group_name>')
+        .with_statistics_config(stats_config)
+  )
+
+You can call the ``get_statistics()`` method of the dataset to fetch metrics for a specific ingestion job.
+
+The ``get_statistics()`` method takes the following optional parameter:
+
+- ``job_id: string``. Id of feature group job
+
+.. code-block:: python3
+
+  # Fetch stats results for a dataset job
+  df = dataset.get_statistics(job_id).to_pandas()
+
+.. image:: figures/stats_1.png
+
+For more details on statistics computation please refer :ref:`Statistics`
+
+
 Get features
 ============
 You can call the ``get_features_dataframe()`` method of the Dataset instance to fetch features in a dataset.
diff --git a/ads/feature_store/docs/source/feature_group.rst b/ads/feature_store/docs/source/feature_group.rst
@@ -150,61 +150,69 @@ Feature store provides an API similar to Pandas to join feature groups together
                 .join(feature_group_c.select(), left_on=['b_1'], right_on=['c_1'])
   query.show(5)
 
-<<<<<<< Updated upstream
 Save expectation entity
 =======================
-With a ``FeatureGroup`` instance, You can save the expectation details using ``with_expectation_suite()`` with parameters
+Feature store allows you to define expectations on data being materialized into feature group instance. With a ``FeatureGroup`` instance, we can save the expectation entity using ``save_expectation()``
 
-- ``expectation_suite: ExpectationSuite``. ExpectationSuit of great expectation
+
+.. image:: figures/validation.png
+
+The ``.save_expectation()`` method takes the following optional parameter:
+
+- ``expectation: Expectation``. Expectation of great expectation
 - ``expectation_type: ExpectationType``. Type of expectation
         - ``ExpectationType.STRICT``: Fail the job if expectation not met
         - ``ExpectationType.LENIENT``: Pass the job even if expectation not met
 
-.. note::
+.. code-block:: python3
 
-  Great Expectations is a Python-based open-source library for validating, documenting, and profiling your data. It helps you to maintain data quality and improve communication about data between teams. Software developers have long known that automated testing is essential for managing complex codebases.
+  feature_group.save_expectation(expectation_suite, expectation_type="STRICT")
 
-.. image:: figures/validation.png
+For more details on expectation please refer :ref:`Feature Validation`
+
+
+Statistics Computation
+========================
+During the materialization feature store performs computation of statistical metrics for all the features  by default. This can be configured using ``StatisticsConfig`` object which can be passed at the creation of
+feature group or it can be updated later as well.
 
 .. code-block:: python3
 
-    expectation_suite = ExpectationSuite(
-        expectation_suite_name="expectation_suite_name"
-    )
-    expectation_suite.add_expectation(
-        ExpectationConfiguration(
-            expectation_type="expect_column_values_to_not_be_null",
-            kwargs={"column": "<column>"},
-        )
+  # Define statistics configuration for selected features
+  stats_config = StatisticsConfig().with_is_enabled(True).with_columns(["column1", "column2"])
 
-    feature_group_resource = (
-        FeatureGroup()
-        .with_feature_store_id(feature_store.id)
-        .with_primary_keys(["<key>"])
-        .with_name("<name>")
-        .with_entity_id(entity.id)
-        .with_compartment_id(<compartment_id>)
-        .with_schema_details_from_dataframe(<datframe>)
-        .with_expectation_suite(
-            expectation_suite=expectation_suite,
-            expectation_type=ExpectationType.STRICT,
-         )
-    )
 
-You can call the ``get_validation_output()`` method of the FeatureGroup instance to fetch validation results for a specific ingestion job.
+This can be used with feature group instance.
+
+.. code-block:: python3
+
+  # Fetch stats results for a feature group job
+  from ads.feature_store.feature_group import FeatureGroup
 
-Statistics Results
-==================
-You can call the ``get_statistics()`` method of the FeatureGroup instance to fetch statistics for a specific ingestion job.
+  feature_group_resource = (
+    FeatureGroup()
+    .with_feature_store_id(feature_store.id)
+    .with_primary_keys(["<key>"])
+    .with_name("<name>")
+    .with_entity_id(entity.id)
+    .with_compartment_id(<compartment_id>)
+    .with_schema_details_from_dataframe(<dataframe>)
+    .with_statistics_config(stats_config)
+
+You can call the ``get_statistics()`` method of the feature group to fetch metrics for a specific ingestion job.
+
+The ``get_statistics()`` method takes the following optional parameter:
+
+- ``job_id: string``. Id of feature group job
 
 .. code-block:: python3
 
   # Fetch stats results for a feature group job
   df = feature_group.get_statistics(job_id).to_pandas()
 
 .. image:: figures/stats_1.png
-=======
->>>>>>> Stashed changes
+
+For more details on statistics computation please refer :ref:`Statistics`
 
 Get last feature group job
 ==========================
diff --git a/ads/feature_store/docs/source/feature_validation.rst b/ads/feature_store/docs/source/feature_validation.rst
@@ -1,25 +1,9 @@
+.. _Feature Validation:
+
 Feature Validation
 *************
 
-Save expectation entity
-=======================
-With a ``FeatureGroup`` or ``Dataset`` instance, we can save the expectation entity using ``save_expectation()``
-
 .. note::
+  `Great Expectations <https://docs.greatexpectations.io/docs/>_` is a Python-based open-source library for validating, documenting, and profiling your data. It helps you to maintain data quality and improve communication about data between teams. Software developers have long known that automated testing is essential for managing complex codebases.
 
-  Great Expectations is a Python-based open-source library for validating, documenting, and profiling your data. It helps you to maintain data quality and improve communication about data between teams. Software developers have long known that automated testing is essential for managing complex codebases.
-
-.. image:: figures/validation.png
-
-The ``.save_expectation()`` method takes the following optional parameter:
-
-- ``expectation: Expectation``. Expectation of great expectation
-- ``expectation_type: ExpectationType``. Type of expectation
-        - ``ExpectationType.STRICT``: Fail the job if expectation not met
-        - ``ExpectationType.LENIENT``: Pass the job even if expectation not met
-
-.. code-block:: python3
-
-  feature_group.save_expectation(expectation_suite, expectation_type="STRICT")
-  dataset.save_expectation(expectation_suite, expectation_type="STRICT")
 
diff --git a/ads/feature_store/docs/source/overview.rst b/ads/feature_store/docs/source/overview.rst
@@ -33,3 +33,5 @@ Oracle feature store is a stack based solution that is deployed in the customer
         - .. image:: https://img.shields.io/badge/delta-2.0.1-blue?style=for-the-badge&logo=pypi&logoColor=white
       * - pyspark
         - .. image:: https://img.shields.io/badge/pyspark-3.2.1-blue?style=for-the-badge&logo=pypi&logoColor=white
+
+  Please contact #oci-feature-store_early-preview for getting your tenancy whitelisted for early access of feature store.
diff --git a/ads/feature_store/docs/source/statistics.rst b/ads/feature_store/docs/source/statistics.rst
@@ -1,3 +1,5 @@
+.. _Statistics:
+
 Statistics
 *************
 
@@ -7,72 +9,3 @@ to derive insights about the data quality.
 .. note::
 
   Feature Store utilizes MLM Insights which is a Python API that helps evaluate & monitor data for entirety of ML Observability lifecycle. It performs data summarization which reduces a dataset into a set of descriptive statistics.
-
-
-Statistics Configuration
-========================
-Computation of statistical metrics happens by default for all the features but you can configure it using ``StatisticsConfig`` object. This object can be passed at the creation of
-feature group or dataset or it can be later updated as well.
-
-.. code-block:: python3
-
-  # Define statistics configuration for selected features
-  stats_config = StatisticsConfig().with_is_enabled(True).with_columns(["column1", "column2"])
-
-
-This can be used with feature group instance.
-
-.. code-block:: python3
-
-  # Fetch stats results for a feature group job
-  from ads.feature_store.feature_group import FeatureGroup
-
-  feature_group_resource = (
-    FeatureGroup()
-    .with_feature_store_id(feature_store.id)
-    .with_primary_keys(["<key>"])
-    .with_name("<name>")
-    .with_entity_id(entity.id)
-    .with_compartment_id(<compartment_id>)
-    .with_schema_details_from_dataframe(<dataframe>)
-    .with_statistics_config(stats_config)
-
-Similarly for dataset instance.
-
-.. code-block:: python3
-
-  from ads.feature_store.dataset import Dataset
-
-  dataset = (
-        Dataset
-        .with_name("<dataset_name>")
-        .with_entity_id(<entity_id>)
-        .with_feature_store_id("<feature_store_id>")
-        .with_description("<dataset_description>")
-        .with_compartment_id("<compartment_id>")
-        .with_dataset_ingestion_mode(DatasetIngestionMode.SQL)
-        .with_query('SELECT col FROM <entity_id>.<feature_group_name>')
-        .with_statistics_config(stats_config)
-  )
-
-Statistics Results
-==================
-You can call the ``get_statistics()`` method of the FeatureGroup or Dataset instance to fetch validation results for a specific ingestion job.
-
-The ``get_statistics()`` method takes the following optional parameter:
-
-- ``job_id: string``. Id of feature group/dataset job
-
-.. code-block:: python3
-
-  # Fetch stats results for a feature group job
-  df = feature_group.get_statistics(job_id).to_pandas()
-
-similarly for dataset instance
-
-.. code-block:: python3
-
-  # Fetch stats results for a dataset job
-  df = dataset.get_statistics(job_id).to_pandas()
-
-.. image:: figures/stats_1.png
diff --git a/ads/feature_store/docs/source/terraform.rst b/ads/feature_store/docs/source/terraform.rst
@@ -6,10 +6,6 @@ Oracle feature store is a stack based solution that is deployed in the customer
 Customer can stand up the service with infrastructure in their own tenancy. The service consists of API in customer
 tenancy using resource manager.
 
-.. note::
-
-    Please contact #oci-feature-store_early-preview for getting your tenancy whitelisted for early access of feature store.
-
 Below is the terraform stack deployment diagram of the feature store resources.
 
 .. figure:: figures/feature_store_deployment.png
