Skip to content

Commit 3f2b878

Browse files
committed
Merge remote-tracking branch 'origin/feature/feature-store-marketplace-operator' into feature/feature-store-marketplace-operator
2 parents 8b5eae3 + eb6eb76 commit 3f2b878

File tree

2 files changed

+39
-0
lines changed

2 files changed

+39
-0
lines changed

ads/feature_store/docs/source/feature_group.rst

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,28 @@ The ``.materialise_stream()`` method takes the following parameter:
145145
- ``write_options``: Extra write options for Spark as key-value pairs.
146146
- Defaults to ``{}``.
147147

148+
.. code-block:: python3
149+
150+
from ads.feature_store.feature_group import FeatureGroup
151+
152+
from ads.feature_store.common.spark_session_singleton import SparkSessionSingleton
153+
from pyspark.sql.types import StructType
154+
155+
# Get the spark session managed by the feature store
156+
spark = SparkSessionSingleton(metastore_id=metastore_id).get_spark_session()
157+
158+
df = spark.readStream \
159+
.option("sep", ",") \
160+
.option("header", True) \
161+
.csv(f"{stream_data_dir}/")
162+
163+
feature_group = FeatureGroup.from_id("<unique_id>")
164+
165+
query = fg.materialise_stream(input_dataframe=df, checkpoint_dir=f"{stream_data_dir}chec")
166+
167+
query.awaitTermination()
168+
169+
148170
.. seealso::
149171
:ref:`Feature Group Job`
150172

ads/feature_store/docs/source/notebook.rst

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,23 @@ Big data operations using OCI DataFlow
4949
- | * Ingestion of data using Spark Magic.
5050
| * Querying and exploration of data using Spark Magic.
5151
52+
Streaming operations using Spark Streaming
53+
##########################################
54+
55+
.. admonition:: Notebook Examples
56+
:class: note
57+
58+
.. list-table::
59+
:widths: 50 50
60+
:header-rows: 1
61+
62+
* - Jupyter Notebook
63+
- Description
64+
65+
* - `Streaming operations with feature store <https://github.com/oracle-samples/oci-data-science-ai-samples/blob/main/notebook_examples/feature_store_streaming_data_frame.ipynb>`__
66+
- | * Ingestion of data using spark streaming.
67+
| * Modes of ingestion: ``COMPLETE`` and ``APPEND``.
68+
5269
LLM Use cases
5370
#############
5471

0 commit comments

Comments
 (0)