add more doc content

guptadivyank · guptadivyank · commit 5950ac163176 · 2023-07-12T17:02:00.000+05:30
diff --git a/ads/feature_store/docs/source/dataset.rst b/ads/feature_store/docs/source/dataset.rst
@@ -142,7 +142,9 @@ The ``.save_expectation()`` method takes the following optional parameter:
 
   feature_group.save_expectation(expectation_suite, expectation_type="STRICT")
 
-For more details on expectation please refer :ref:`Feature Validation`
+.. seealso::
+
+    :ref:`Feature Validation`
 
 Statistics Computation
 ========================
@@ -177,7 +179,7 @@ You can call the ``get_statistics()`` method of the dataset to fetch metrics for
 
 The ``get_statistics()`` method takes the following optional parameter:
 
-- ``job_id: string``. Id of feature group job
+- ``job_id: string``. Id of dataset job
 
 .. code-block:: python3
 
@@ -186,7 +188,9 @@ The ``get_statistics()`` method takes the following optional parameter:
 
 .. image:: figures/stats_1.png
 
-For more details on statistics computation please refer :ref:`Statistics`
+.. seealso::
+
+    :ref:`Statistics`
 
 
 Get features
diff --git a/ads/feature_store/docs/source/feature_group.rst b/ads/feature_store/docs/source/feature_group.rst
@@ -168,7 +168,9 @@ The ``.save_expectation()`` method takes the following optional parameter:
 
   feature_group.save_expectation(expectation_suite, expectation_type="STRICT")
 
-For more details on expectation please refer :ref:`Feature Validation`
+.. seealso::
+
+   :ref:`Feature Validation`
 
 
 Statistics Computation
@@ -212,7 +214,9 @@ The ``get_statistics()`` method takes the following optional parameter:
 
 .. image:: figures/stats_1.png
 
-For more details on statistics computation please refer :ref:`Statistics`
+.. seealso::
+
+    :ref:`Statistics`
 
 Get last feature group job
 ==========================
diff --git a/ads/feature_store/docs/source/feature_validation.rst b/ads/feature_store/docs/source/feature_validation.rst
@@ -3,7 +3,52 @@
 Feature Validation
 *************
 
+Feature validation is the process of checking the quality and accuracy of the features used in a machine learning model. This is important because features that are not accurate or reliable can lead to poor model performance.
+Feature store allows you to define expectation on the data which is being materialized into feature group & dataset. This is achieved using open source library Great Expectations.
+
 .. note::
-  `Great Expectations <https://docs.greatexpectations.io/docs/>_` is a Python-based open-source library for validating, documenting, and profiling your data. It helps you to maintain data quality and improve communication about data between teams. Software developers have long known that automated testing is essential for managing complex codebases.
+  `Great Expectations <https://docs.greatexpectations.io/docs/>`_ is a Python-based open-source library for validating, documenting, and profiling your data. It helps you to maintain data quality and improve communication about data between teams. Software developers have long known that automated testing is essential for managing complex codebases.
+
+
+Expectations
+============
+An Expectation is a verifiable assertion about your data. You can define expectation as below:
+
+.. code-block:: python3
+
+    from great_expectations.core.expectation_configuration import ExpectationConfiguration
+
+    # Create an Expectation
+    expect_config = ExpectationConfiguration(
+        # Name of expectation type being added
+        expectation_type="expect_table_columns_to_match_ordered_list",
+        # These are the arguments of the expectation
+        # The keys allowed in the dictionary are Parameters and
+        # Keyword Arguments of this Expectation Type
+        kwargs={
+            "column_list": [
+                "column1",
+                "column2",
+                "column3",
+                "column4",
+            ]
+        },
+        # This is how you can optionally add a comment about this expectation.
+        meta={
+            "notes": {
+                "format": "markdown",
+                "content": "details about this expectation. **Markdown** `Supported`",
+            }
+        },
+    )
+
+Expectations Suite
+============
+
+Expectation Suite is a collection of verifiable assertions i.e. expectations about your data. You can define expectation suite as below:
 
+.. code-block:: python3
 
+    # Create an Expectation Suite
+    suite = context.add_expectation_suite(expectation_suite_name="example_suite")
+    suite.add_expectation(expect_config)
diff --git a/ads/feature_store/docs/source/overview.rst b/ads/feature_store/docs/source/overview.rst
@@ -15,6 +15,10 @@ Oracle feature store is a stack based solution that is deployed in the customer
 - ``Dataset``: A dataset is a collection of feature that are used together to either train a model or perform model inference.
 - ``Dataset Job``: Dataset job is the execution instance of a dataset. Each dataset job will include validation results and statistics results.
 
+.. important::
+
+      Prerequisite : Please contact #oci-feature-store_early-preview for getting your tenancy whitelisted for early access of feature store.
+
 .. important::
 
   The OCI Feature Store support following versions
@@ -32,6 +36,4 @@ Oracle feature store is a stack based solution that is deployed in the customer
       * - delta-spark
         - .. image:: https://img.shields.io/badge/delta-2.0.1-blue?style=for-the-badge&logo=pypi&logoColor=white
       * - pyspark
-        - .. image:: https://img.shields.io/badge/pyspark-3.2.1-blue?style=for-the-badge&logo=pypi&logoColor=white
-
-  Please contact #oci-feature-store_early-preview for getting your tenancy whitelisted for early access of feature store.
+        - .. image:: https://img.shields.io/badge/pyspark-3.2.1-blue?style=for-the-badge&logo=pypi&logoColor=white
diff --git a/ads/feature_store/docs/source/statistics.rst b/ads/feature_store/docs/source/statistics.rst
@@ -3,9 +3,41 @@
 Statistics
 *************
 
-Feature Store provides functionality to compute statistics for feature groups & datasets and persist them along with the metadata. These statistics can help you
-to derive insights about the data quality.
+Feature Store provides functionality to compute statistics for feature groups as well as datasets and persist them along with the metadata. These statistics can help you
+to derive insights about the data quality. These statistical metrics are computed during materialisation time and persisting with other metadata.
 
 .. note::
 
-  Feature Store utilizes MLM Insights which is a Python API that helps evaluate & monitor data for entirety of ML Observability lifecycle. It performs data summarization which reduces a dataset into a set of descriptive statistics.
+  Feature Store utilizes MLM Insights which is a Python API that helps evaluate and monitor data for entirety of ML Observability lifecycle. It performs data summarization which reduces a dataset into a set of descriptive statistics.
+
+The statistical metrics that are computed by feature store depend on the feature type.
+
+Metrics for categorical data
+
+- Count
+- TopKFrequentElements
+- TypeMetric
+- DuplicateCount
+- Mode
+- DistinctCount
+
+Metrics for numerical data
+
+- Skewness
+- StandardDeviation
+- Min
+- IsConstantFeature
+- IQR
+- Range
+- ProbabilityDistribution
+- Variance
+- TypeMetric
+- FrequencyDistribution
+- Count
+- Max
+- DistinctCount
+- Sum
+- IsQuasiConstantFeature
+- Quartiles
+- Mean
+- Kurtosis
